Thanks for the fast reply and sorry for my late response. We have a scraping on 15 seconds. My query interval is 30 min. So I should have data. And the Grafana dashboard below where I add difference(changed from delta) on legend values I see what I want. This difference from the Grafana panel is exactely what I am looking for and thought delta would give me.
Is there an alternative to delta? Like first-last? On Saturday, July 1, 2023 at 3:07:22 PM UTC+2 Ben Kochie wrote: > Grafana defaults to a "minimum step" of 15 seconds. I bet your scrape > interval is longer than 15 seconds, so "$__interval" is matching no data. > > You can either adjust the datasource minimum step to your shortest scrape > interval, or set the "min step" in the query options to match the scrape > interval of your metric. > > On Sat, Jul 1, 2023 at 3:04 PM Helena Lundell <[email protected]> wrote: > >> I am trying to identify an alert for an increasing Kafka Lag. The metrics >> are in prometheus and I use Grafana 9.4 for visualisation. >> >> In Grafana I have a dasboard where I can see that the lag is increasing >> or decreasing. I have choosen to display the delta in the legend and it >> shows a number that does not really make sense, but it shows something. >> >> This is the base promql: >> avg by(consumergroup, topic, cluster) >> (kafka_consumergroup_lag{namespace="ns-kafka-int", >> consumergroup=~".*$container", cluster="$cluster"}) >> >> When I add delta to this query I get no data: >> avg by(consumergroup, topic, cluster) >> (delta(kafka_consumergroup_lag{namespace="ns-kafka-int", >> consumergroup=~".*$container", cluster="$cluster"}[$__interval])) >> >> So adding delta gives me no result even though when I exclude delta I can >> see that the metric has been changing. >> >> The metric is a gaugue. >> >> Question: >> Am I doing something wrong? >> Can I trust the delta from the Grafana dashboard(someone might know)? >> Can I expect a positive or negative value to identify and increase? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/63cf733e-87bd-4530-9531-5a3a29556d8an%40googlegroups.com.
