Gregory's Server@ProPublica @derekwillis I feel like it should be possible to build a o tool to do this
|
4 comments
 @derekwillis @ProPublica 10k accounts, 10 tweets/day, 100k tweets/day, what, 1k of data per tweet, less than 500 GB over 10 years, I feel like it wouldn't more than $100/mo in cloud services, would it? I don't have any sense of the budget available for something like this, though. @JetForMe @ProPublica it's complicated, Politwoops relies on the streaming API, which means it has to run 24/7, because in order to know which tweets have been deleted you have to have copies of them first (the notification is just the ID). That latter bit is what's currently broken - Twitter isn't sending those notifications. So we would have to, I guess, go back and check if tweets had been deleted, which is possible but you lose time context. Also, ppl delete stuff after a long time. @JetForMe @ProPublica So you could build a system that does all of that, absent the timeliness factor, but it would be difficult to say that you'd be capturing all of the deletions. |
@JetForMe @ProPublica it is possible; the problems are scale (thousands of accounts to track) and timeliness. And, not insignificantly, cost. Twitter could just run this themselves, tbh, but I understand why they don't.