UPDATE: I found out the cause of an extremely esoteric bug / issue with the Pushshift API. Once every 10 or so seconds, the main Verizon router will take 5+ seconds to respond to simple pings. Resetting the router fixes the issue for another 24-36 hours, but eventually it will
Conversation
This Tweet was deleted by the Tweet author. Learn more
Replying to
I was actually in the middle of an Overwatch game and every ten seconds, my game would get all out of whack. I then started pinging the main router and sure enough, once every ten seconds, it would take 5-6 seconds to respond. So that would affect every API behind the router.
Replying to
I don't know why the router is fine for 24-36 hours after resetting it but my hunch is that there is an internal counter somewhere in the code and it hits the limit for a 32 bit counter for the number of packets going through the router.
The router is sending and receiving...
2
Replying to
a few terabytes of data every day so it's handling about 50x the normal usage for this router probably.
1
This Tweet was deleted by the Tweet author. Learn more
Replying to
The goal going into the new year is to vastly improve all of the API ingest pipelines and upgrade the cluster to the latest version which should help tremendously.