Thanks for the reply. > The underlying requirement you have is, I think, given a collection of recorded rate[5m] values, can you turn this into a rate[1d] ?
Yes, that’s correct. Regarding the 5m vs. 4m: I tried to adjust for that by increasing the rate window by one scrape interval (in this case 15s). The result is still quite a ways off: avg_over_time(rate(requests_total[315s])[10m:5m])*(10*60) => 1820 (expected 2153.84) I also tried this, to include more “slices” in the subquery. Still off: avg_over_time(rate(requests_total[75s])[10m:1m])*(10*60) => 2045.45 (expected 2153.84) I suspect there is some mathematical property I can’t articulate yet that means this won’t ever be 100% accurate, even if we adjust for what promql is doing. I interpreted your last comment as suggesting that the short time period and choice of rate windows was the problem. While those things do affect the result (see above), I can’t find a combination that produces the expected result. Could you elaborate what you meant? On Thursday, June 29, 2023 at 12:27:43 AM UTC-7 Brian Candler wrote: > Not in the presence of counter resets, no. > > > My full question, reason for the question, and experiment is here: > https://github.com/dsabsay/prom_rates_and_counters/blob/main/README.md > > "In other words, can one get the equivalent of increase(some_metric[1d]) by > using the output of a recording rule like rate(some_metric[5m])? > > I think that increase(...) doesn't work in the way you think it does. > > increase(...) and rate(...) are the same thing, only differing by a factor > of the time window. That is: increase(some_metric[1d]) is *exactly* the > same as rate(some_metric[1d])*86400. > > To find the exact difference between a metric now and 1d ago, you can use > some_metric - some_metric offset 1d > > However that does not work across counter resets, for obvious reasons. > > The underlying requirement you have is, I think, given a collection of > recorded rate[5m] values, can you turn this into a rate[1d] ? I think the > avg_over_time() of those rates is the best you can do. If these are 5m > rates, then you'd want 5 minute steps: avg_over_time( [ xx : 5m] ) > > But you are testing this over very short time periods (10m) and therefore > it's not going to be exact. In particular, rate([5m]) takes the rate > between the first and last data points in a 5 minute window. This means > that if you are scraping at 1 minute intervals, you're actually calculating > a rate over a 4 minute period. > > On Thursday, 29 June 2023 at 07:22:43 UTC+1 Daniel Sabsay wrote: > >> Is it possible to accurately calculate original counts from pre-recorded >> rates? >> My experiments suggest the answer is no. But I’m curious to get other >> perspectives to see if I’ve overlooked something or if there is a more >> effective way to approach this. >> >> My full question, reason for the question, and experiment is here: >> https://github.com/dsabsay/prom_rates_and_counters/blob/main/README.md >> >> Thanks! >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/281dacd1-f71e-443f-8215-89247b6c9ea1n%40googlegroups.com.
