Each day, Pushshift is now ingesting, indexing and storing over half a billion social media objects. The eventual goal is to be able to process up to one billion objects daily for researchers to use to better understand how social media platforms influence society.
Conversation
Replying to
The scale you're at is astounding. My system (using a small portion of pushift's dataset) is absolutely chugging at only 8 billion objects 😰
1
2
Replying to
That's awesome! Really happy to hear people are benefiting from what we do.
1
1
Show replies