Anil, Flink Histograms are actually summaries .. You need to override the Prometheus Histogram class provided to write it into different buckets to Prometheus .. Then you can write prom queries to calculate different quantiles accordingly ... Checkpointing The histograms is not a recommended option/solution
Thanks, Prasanna. On Tue, 5 Apr 2022, 01:26 Anil K, <sendto.ani...@gmail.com> wrote: > Hi, > I was doing some experimentation using Histograms, had a few questions > mostly related to fault tolerance and restarts. I am looking for a way to > calculate p95 over 30days. Since histograms are pushed as a summary into > prometheus, will not be able to do the aggregation for 30 days at > Prometheus' side from smaller windows. Also there is a high chance that > the job may restart in between. > > So Are Histograms Checkpointed ? I am looking for a very large > SlidingTimeWindowReservoir of 30days and I don't want to lose the histogram > updates till then and start from scratch after restart. > > If Histograms are not checkpointed and are reset, are there any other > possible ways to calculate Histograms/P95 on a large window like 30days? > > Thanks & Regards > Anil > > > > >
