Conversation

I have created a new Github repo for election data related scripts. The repo is public and located here: github.com/pushshift/US_E So far, the script (Python) pulls data for the 2020 election for every county in the US. #Election2020 #bigdata #elections

github.com

GitHub - pushshift/US_Election_Data: Code to grab election data from CNN's election data API

Code to grab election data from CNN's election data API - GitHub - pushshift/US_Election_Data: Code to grab election data from CNN's election data API

12:21 AM · Nov 15, 2020Twitter Web App

Replying to

What I will try to do is locate as much election data as possible and try to locate historical data and then put all of the data in a Pushshift API endpoint. This should be helpful for a number of election related projects. Please give me suggestions if you have them!

Replying to

It should be possible to set up an index in elasticsearch for this data and also for historical election data. I just need to locate a few reliable sources for historical election data and then standardize / normalize the fields and create an index mapping in ES.

Replying to

I just added support for a simple sqlite3 database. So if you run the script, it will create a sqlite3 database and then store the data in a table called "county". The county table is a simple table with the following fields: id (fips county id), retrieved_utc, state and data.

Replying to

INFO: After running the script, there were a total of 4,634 rows in the county table.