I am now crawling URLs of all videos uploaded to Parler. Sequentially from latest to oldest. VIDXXX.txt files coming up, 50k chunks, there will be 1.1M URLs total: https://donk.sh/06d639b2-0252-4b1e-883b-f275eff7e792/ … This may include things from deleted/private posts.
-
-
I'd estimate the total size for this would be ~80TB, 4TB per chunk. It's S3/CloudFront so as much bandwidth as you can throw at it.
Show this thread -
The crawl is now complete. 1098552 video URLs
Show this thread -
somebody shared this with me for mirroring the videos to S3 fasthttps://github.com/tkellen/parlerdump …
Show this thread -
i should probably warn you that if you do this, it will cost you $3k in S3 egress fees to get it back out. but now we know it's 30TB total. https://twitter.com/tkellen/status/1348403709592674310 …
This Tweet is unavailable.Show this thread
End of conversation
New conversation -
-
-
Doing the god’s work
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
You can store media files for free at http://archive.org , which might also be a nice secure place for this content.
-
yes, that's the best place to put it
- Show replies
New conversation -
-
-
I could store ~half a GB for you on a removable drive. Warning, I'm in rural New Mexico so my download bandwidth isn't great. Early AM after the gamers all conk out is best time to download.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
Meiklejohnian absolutist. free speech as in free-for-everyone.
mutuals: