Pushshift will be rolling out some new API endpoints (especially for Reddit data). The current production API endpoint is unfortunately behind quite a bit due to limitations in the ingest code. I will be releasing a new endpoint in the next few days that will have a redesigned
Conversation
ingest pipeline that can handle the increase in Reddit volume so that it won't fall behind by more than 20-30 seconds. This is a high priority for researchers who depend on near real-time data. Eventually all data will be migrated over to the new endpoint along with filling any
1
13
data gaps that currently exist. The new endpoint for comments and submissions will be announced in the next few days so stay tuned -- we're working hard to improve all aspects of our ingest pipelines.