JSON Baumgartner on Twitter: "@flashman When the API goes live the old methods will still work. In the meantime they can be aggregated using Elasticsearch style aggs. There are some examples in the /r/pushshift subreddit and I'm also happy to show you any particular ones." / Twitter

If you need massive amounts of Reddit data quickly, the new beta API that is up allows getting comments and submissions 5 to 10 times faster than the current API. Here's an example of getting 25,000 of the latest comments from the politics subreddit. #bigdata #datascience

0:16

2K views

4

16

87

JSON Baumgartner

@jasonbaumgartne

Jul 27, 2019

Here is the code that you can run to benchmark on your end (and the endpoint to use to get data). This endpoint accepts all Elasticsearch aggregations and search features. reddit.com/r/pushshift/co #api #datascience

reddit.com

New Beta API vs Current API Benchmark

Here's an exciting benchmark. The goal is to simply collect the previous 50,000 comments from the AskReddit subreddit in batches of...

1

2

16

Replying to

Feature request: emoji search

1

Replying to

Already got you covered. dev.pushshift.io/rc/_search/?pr:🍔

1

Replying to

and

You may need to copy and paste that ... the hamburger should come right after body:

1

Replying to

Hmm. You've changed the syntax for filtering and aggregation? "aggs=subreddit" and "subreddit=funny" are throwing errors for me e.g. dev.pushshift.io/rc/_search/?pr

1

Replying to

and

Or not changed, but I just don't understand how to do it in that format.

1

JSON Baumgartner

@jasonbaumgartne

Replying to

@flashman

When the API goes live the old methods will still work. In the meantime they can be aggregated using Elasticsearch style aggs. There are some examples in the /r/pushshift subreddit and I'm also happy to show you any particular ones.

1:02 PM · Jul 27, 2019Twitter Web App

Conversation