Nice to read if you ever wondered why our Histogram is so different and what is that MixMaxCounter we talk about: http://kamon.io/core/metrics/instruments/ …
@kamonteam @giltene guessing this is a performance vs precision tradeoff but curious if anyone's done the benchmarking.
-
-
@avibryant@kamonteam actually, it has perfect percentile precision, and O(1) trivially fast speed. -
@avibryant@kamonteam It trades value precision (e.g. to N decimal points) and space to gain the speed and percentile precision qualities. -
@giltene@kamonteam so, one could imagine a pathological case where the variance was on the order of 1/1000 the value. -
@giltene@kamonteam in that case the HdrHistogram would tell you very little beyond the mean and the fact of the very low variance. -
@avibryant@kamonteam Yup. If you configure it with 3 decimal point accuracy, all numbers equal to within that will look the same. -
@avibryant@kamonteam But that's not an "interesting" problem area for measuring latency percentiles… -
@giltene@kamonteam yep. I'm coming from a machine learning context, where being able to eg find that "DC offset" on the fly is more useful. -
@avibryant@kamonteam In this case it's trivial at the start: E.g. take the average of the first 100 results in the stream before feeding. - 10 more replies
New conversation -
-
-
@avibryant@kamonteam Haven't measured q-digest or t-digest, but HdrHistogram can do 100M-200M/sec. Pretty sure it's dramatically faster. -
.
@avibryant@kamonteam HdrHistogram eliminates the need for binning. I only recently realized that it's actually a new data structure. -
.
@avibryant@kamonteam The key "newness" is that HdrHistogram provide precise quantiles for estimated values. As opposed to the reverse. -
.
@avibryant@kamonteam Also, HdrHistogram focuses on extreme streaming speed (fixed time, fixed time). Not on percentile query speeds.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.