@aphyr you were referenced in our recent write up about scaling NSQ: https://segment.com/blog/scaling-nsq/ …
-
-
Replying to @brentsummers
I was evaluating NSQ a while back but wound up not using it because... if you lose a node, you lose all the messages on it.
1 reply 0 retweets 0 likes -
Replying to @aphyr @brentsummers
With that in mind, I'm curious: what kind of workload needs an on-disk queue but only requires best-effort message delivery?
2 replies 0 retweets 0 likes -
Replying to @brentsummers @aphyr
ah yes. We talk about it a little in the article, but you're right, the delivery guarantees aren't super strict.
1 reply 0 retweets 0 likes -
we're planning on moving a lot over to Kafka for the replication. Though NSQ has been rock-solid operationally
1 reply 0 retweets 0 likes -
we're able to mitigate a bit with a raw archive on S3 and flushing to EBS. That said, it's def possible to lose writes
1 reply 0 retweets 0 likes -
Replying to @calvinfo @brentsummers
ah, I gotcha, so you journal to S3 and can recover when an EBS volume fails? Seems sensible. :)
1 reply 0 retweets 0 likes
haha yeah, there's a lot to be improved. That's where Kafka will come into play :)
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
