Using #bigquery, I can now quickly see the delta for my #reddit ingest between the time a comment was made to Reddit and when I ingested it. The delta field is in seconds. You can see that a majority of comments are ingested within one second!
#datascience #bigdata
Conversation
Replying to
Using #bigquery, it will be very easy to take occasional "health checkups" of the Pushshift API by very quickly scanning and grouping data (something BigQuery excels at)
1
