I'd like to quickly talk about #pushshift API failover in the event there is a disruption on the main servers. My thinking right now considering the restrictions imposed by financing is to have a backup server located in the cloud () that will always hold the
Conversation
previous few months of data. So in the event of a critical failure, the API will still be able to handle requests for data within the past 90 days (which is usually the bulk of requests). I have api.pushshift.io/meta which shows the current rate-limits. I could add a field
Replying to
that would signify if a failover occurred and that it was using the backup server to serve API requests. Ideally, I'd have full redundancy but that isn't possible at this time given financial restraints. I think it is a good compromise as it would keep the service available.
1