@avibryant @kamonteam It trades value precision (e.g. to N decimal points) and space to gain the speed and percentile precision qualities.
@giltene @kamonteam this isn't an academic question - I'm thinking in particular about using it in MapReduce, where the nodes have no comm.
-
-
@avibryant@kamonteam What would you consider an acceptable footprint for an "accurate" histogram, BTW? -
@avibryant@kamonteam E.g would 0.5MB be acceptable for 0.01% accuracy across 12 [decimal] orders of magnitude? How about 5MB for 0.001%? -
@giltene@kamonteam those footprints are definitely acceptable. -
@avibryant@kamonteam Yeh, figured that for MapReduce stuff they would be. I'm used to low 10s of KB, +/- 1% on value, but precise 99.9999%+ -
@giltene@avibryant jumping in late and Gil already said what needed to be said :). We don’t have experience with q-digest either, but... -
@giltene@avibryant … we can say for sure that the HdrHistogram is by far the most advanced and performant histogram we could have found.
End of conversation
New conversation -
-
-
@avibryant@kamonteam You just need to communicate the offset along with each histogram, and adjust for it when you add. -
@giltene@kamonteam there's some devil in the details there, because the bins won't line up nicely. -
@avibryant@kamonteam no bins (in the regular sense) to worry about. It's will all good the within the integer accuracy level you choose. -
@giltene@kamonteam I guess as long as you are always adjusting values up (possibly losing res) when you add, it should work out, yeah.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.