That 80 gig compressed file is just for tweets from an 18 hour period (if I remember correctly -- it was around 18-24 hours). I was ingesting around 2,000+ tweets for each second in the timeline.
1
1
You’re unable to view this Tweet because this account owner limits who can view their Tweets. Learn more
Yeah -- this method is great when researching specific time periods (also could be used to do a full space search after Trump makes a tweet to see if the same accounts / bots retweet, etc.) But for keyword search, this wouldn't be helpful. I am exploring ways to find methods ..
The big pain in the ass limitation with the Twitter API is with the user timelines only going back 3,200 tweets. I'm trying to find ways around that. If I discover something, I'll write it up.