Update for #reddit data. Pushshift is now in the process of compressing several months of comment data. We will also be adding additional fields including the author creation date that will provide additional metadata for research that looks at user engagement by account age.
Conversation
Over the next few months, Pushshift will be updating many components of its core infrastructure that will greatly reduce latency and improve ingest delays due to Reddit spam.
The longer term goals include making the API much more reliable for researchers. Exciting updates soon!