Pushshift will be adding a full 4chan API in the next several weeks as well as creating daily / monthly dumps for 4chan data for researchers to use.
Since 4chan has a lot of controversial posts, we cannot provide media dumps along with the textual data. If you are doing ...
Conversation
Replying to
Can you make sure to preserve the id/flag fields in forums like /pol/?
4chan has an annoying habit of tossing that information from the JSON once the thread locks. My existing collection notebook needs a scraper module as a fallback. 4plebs isn't that friendly to API requests.
1
2
Replying to
Thanks so much for helping with this. Can you give me more info on the flag fields? What exactly are in these fields?
Replying to
Oh sure. Each board on 4chan can have its own fields that are to some extent set by the board admins, eg /pol/, /bant/, and /biz/ have id fields that are consistent within each thread (but not across threads).
/pol/ has country and board (meme) flags...
1
in the JSON these are country (2 letter code), country_name (string), board_flag (2 letters), and flag_name
They pull from 2 images hosted at the CDN:
1
Show replies