An idea that I've been thinking about implementing is designing endpoints for other data engineers that will allow them to send data to Pushshift for permanent archival and indexing. The goal being to help crowd source data ingest projects and create a centralized repo for data.
Conversation
Other data engineers could use these endpoints to POST data they have collected and send that data in bulk to Pushshift where it can then be indexed and distributed to other researchers.
This would allow rapid prototyping of new projects that involve big data.