@kamonteam @giltene do you have a sense of how HDRHistogram performs vs q-digest or t-digest where the binning dynamically responds to data?
@giltene @kamonteam bit trickier in a distributed context; each node will have a slightly different average, then you have to reconcile.
-
-
@avibryant@kamonteam Reconciling isn't hard. Each node can produce their own histogram, and they are additive later. -
@giltene@kamonteam right, but if they each have a different offset it gets harder. -
@giltene@kamonteam this isn't an academic question - I'm thinking in particular about using it in MapReduce, where the nodes have no comm. -
@avibryant@kamonteam What would you consider an acceptable footprint for an "accurate" histogram, BTW? -
@avibryant@kamonteam E.g would 0.5MB be acceptable for 0.01% accuracy across 12 [decimal] orders of magnitude? How about 5MB for 0.001%? -
@giltene@kamonteam those footprints are definitely acceptable. -
@avibryant@kamonteam Yeh, figured that for MapReduce stuff they would be. I'm used to low 10s of KB, +/- 1% on value, but precise 99.9999%+ -
@giltene@avibryant jumping in late and Gil already said what needed to be said :). We don’t have experience with q-digest either, but... - 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.