We're excited to share some of the architecture we're using to scale message delivery @segment. Centrifuge has delivered 500 billion messages, and continues to scale out.https://twitter.com/segment/status/999305866514874368 …
-
-
Hey, how do you avoid node going OOM(even though processing is really fast < 100ms), also when you reschedule/retry jobs they still remain in memory ? that way there would be lot more chances of node going OOM when an outbound API is degraded/down(because reschedule/retry) ?
-
Also in case of high load, do you redistribute the in memory jobs as well ? also stop the node from accepting anymore jobs or something ?
-
Sorry if my questions are naive/dwelling too much, I landed on this a little late, just curious, saw the HN thread as well(but something around this wasnt there, or maybe what I'm asking is naive).
-
Also, great work, inspiring in many ways!
-
Thanks for the kind words. The Directors actively manages memory pressure, and only keep recently active jobs resident in memory. Everything else ends up in MySQL.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.