Conversation
You will need to switch from legacy SQL on the Google BQ dashboard. Just try to do a simple select:
SELECT * FROM `pushshift.rt_reddit.comments` LIMIT 10
The dashboard should be located here: bigquery.cloud.google.com
1
Replying to
That's great news. The feed should be very close to real-time (a 1-3 second delay at most usually).
1
Replying to
So is the idea to provide recent comments only, or will you be pulling in the backlog too and letting people utilise partitions?
1
Replying to
I am starting the real-time ingest now so going forward, people can see what's going on with Reddit on an hourly or daily basic (the table is partitioned by day). There is also the monthly data tables that include more accurate scores (since they have had time to settle)
This isn't meant for historical research -- more along the lines of hourly or daily recaps and analysis going forward.
1
Replying to
Why would I use this over, say, the PushShift API? Because I prefer BQ?
1
1
Show replies