Over 6.5 million politically based tweets captured so far and I've gone back less than three days. Over 35 GB of data so far.
Conversation
Replying to
Unfortunately I don't think you'll be able to determine anything wrt removals/censorship by twitter as a platform on past content correct? Still should be an interesting project.
1
Replying to
I can only crawl 10 days backwards, but once I get all the historical data I can, when I start trailing back up to the present, I can immediately start grabbing things before they get removed unless Twitter has a queue system like Reddit. I don't think they do though.