JSON Baumgartner on Twitter: "@KelseyEGonzalez @jw_lockhart (You will also want to use Felipe's tables for the most recent data)" / Twitter

I keep running into a bug when scraping #reddit comments with #Python 's #PRAW package. Anyone in my network able to help me debug? #computationalsocialscience #soctwitter

1

3

Replying to

Assuming you already checked, but just in case, do the public archives of "all" comments meet your needs? E.g.

@jasonbaumgartne

's files.pushshift.io/reddit/comments

1

Replying to

and

Or the "real-time" / SQL interface in bigquery?

pushshift.io

Using BigQuery with Reddit Data - pushshift.io

Using BigQuery with Reddit data is a lot of fun and easy to do, so let’s get started. You’ll want to start by setting up a BigQuery project if you don’t already have one. Google gives 1TB (one...

1

Kelsey E. Gonzalez, PhD

Replying to

and

It does, I just was invested in using PRAW because I had already put so much time into trying to get it fully functioning by the time someone mentioned it. It's looking like I'll need to transition..

1

Replying to

and

Transitioning tools mid project sucks. :(

1

Kelsey E. Gonzalez, PhD

Replying to

and

Exactly, and for some reason I see using python for the scraping as being higher in cultural capital, but at the end of the day its not working and I need to use a resource that works.

2

1

Replying to

and

Are you working with all subreddits or just certain one? Happy to help.

1

2

Kelsey E. Gonzalez, PhD

Replying to

and

Two different datasets: one with all threads (+ thread comments collapsed as a thread corpus) for about 15 specific subreddits over a pre-defined time period second very similar corpus structure, but pulling threads that include specific keywords from 4 more general subreddits

1