I'm starting to think of some different approaches to data collection and analysis that would benefit the research community. I'd like to set up a modular system / API that allows trusted parties to send data objects to Pushshift from any source. Then any data added becomes
Conversation
public for other researchers to consume. People who add data can then use a suite of analysis tools to analyze the data and share their analysis with others. It would be very modular so that each layer can be used by researchers.
Some researchers might just want access to raw
1
5
data. Others might want access to aggregated views of the raw data. Still others might want to use higher level tools like bot detection algos, anomaly detection, etc. based on the raw data.