The new #reddit ingest code appears to be stable. I've made some additional tweaks to improve the latency. Attached is a picture showing the latency (in seconds) and the percentage of comments retrieved within that time. A majority of comments are fetched within 5 seconds or less
Conversation
Replying to
Do you have a comparison image for the previous implementation? Are you using python?
1
1
Replying to
Yes I am using Python. The previous implementation was consistently behind by hours so it's a huge leap forward. Unfortunately, I don't have any stats right now for the current production implementation.