Hi Aditya, Thank you for the KIP.
A few questions: NS1: how were the metrics picked? There are other latency metrics as well, which are not in the list, for example NS2: Do I understand it correctly that the histograms will be available only via a custom exporter, and not via JMX? Should we consider adding percentiles? NS3: It looks like the KIP proposes cumulative histograms that never reset. If we want to calculate percentiles, making them useful will be a significant challenge. Applications can run for weeks and months. If we keep counting without resets, a p99 calculated over a month isn't very useful. I understand the challenge with custom exporters, but maybe we can rework the idea to let exporters get a predictable delta, so we don't need to keep the history forever? Then the percentiles can be calculated for the configured window, and the future KIP-714 work will be easier as well. NS4: Exponential(Geometrical) histograms have a nice property: each bin at one scale is exactly two bins at the next finer scale. Right now, the KIP proposes having a max, which is very hard to set correctly for latencies. Have you considered not having a cap and automatically adjusting the scale depending on the data? On Mon, Oct 26, 2026 at 11:54 PM Aditya Kousik via dev <[email protected]> wrote: > > Hi all, > > I'd like to start a discussion on KIP-1387, which adds a stat that counts > latency and size values into fixed bins and gives the bin counts to the > metrics reporter. > > KIP link: https://cwiki.apache.org/confluence/x/XwKDGw > > Client latency and size metrics today are an average and a maximum over a > 30-second window. We don’t get a view into how the values are spread, a p99 > can't be derived from them. A percentile computed inside the client can't be > combined across clients or across windows whereas bin counts can. A reporter > can publish them as a Prom or OTel histogram, and a backend query can compute > any percentile, over any period across the whole fleet. > > The KIP adds DistributionStat, whose value is a DistributionSnapshot of > cumulative bin counts. Seven distributions are registered at INFO next to the > existing -avg and -max metrics, with more at DEBUG. Existing metrics don't > change, and JmxReporter skips distributions, so a client reporting over JMX > sees nothing new. > > Looking forward to your feedback. > > Thanks, Aditya > >
