Due to popular demand, I am now creating thorough documentation for the Pushshift search API:
github.com/pushshift/api
Conversation
Replying to
As an example, to find comments posted to Reddit related to Houston over the past 4 hours and find relevant posts: api.pushshift.io/reddit/search/
1
2
Replying to
Yep! It's intended to be a compliment to PRAW. PRAW is a great module, so hopefully this adds a lot of functionality to it.
1
Replying to
Do you plan to add an offset/page function, or should we rely using "before" with created_utc if we want more pages of results?
1
Replying to
Using the before / after parameters works best for the time being. Do you think adding pagination to the API would be helpful?
2
Replying to
BigQuery would probably be more sane and performant (just my guess - not sure though!)
2
Replying to
BigQuery does work better in some use cases, although I have worked hard to get the response times down for the API. For most simple
searches, the response time is usually under 100ms. I agree with you that the pagination is worth pursuing so I'll add it to the list of
1
features to be added to the API. Please feel free to recommend more using Github if you like. I do appreciate your feedback!
1
1
Show replies