Conversation

Replying to

Do you want to restrict to a specific timeframe? We could look at the last month's worth of posts and see what the top X subreddits were based on number of posts, put all the post ids into an array and then get X random out of them. Would be easier if the time-frame was ...

Replying to

I think previous month would be perfect.

Replying to

I'm about to create those dumps shortly -- they're almost done. Would Friday work for delivery?

Replying to

and

Reservoir sample that shit

Replying to

and

Wow, TIL! Thanks Lewis! Very cool sampling method

Replying to

and

That's very cool and good to know!

Replying to

and

Hey

, any luck with that dataset? Thanks!

Replying to

and

Hey

if you’re happy with sampling from the existing pushshift comment dumps I should be able to help with this. Do you need entire comment threads or just n individual comments?

Replying to

and

Thanks so much

, appreciate the offer! I wouldn't want to burden you with this sampling / data wrangling task though... It's a bit of fiddly/tricky problem. [1/2]

Replying to

and

We want to get ~1000 posts from popular subreddits, but sample in such a way that we the posts contain a mix of highly downvoted, highly upvoted, and 'controversial' comments. But perhaps if you're interested in a collaboration, let me know!