I am trying to identify an alert for an increasing Kafka Lag. The metrics
are in prometheus and I use Grafana 9.4 for visualisation.
In Grafana I have a dasboard where I can see that the lag is increasing or
decreasing. I have choosen to display the delta in the legend and it shows
a number that does not really make sense, but it shows something.
This is the base promql:
avg by(consumergroup, topic, cluster)
(kafka_consumergroup_lag{namespace="ns-kafka-int",
consumergroup=~".*$container", cluster="$cluster"})
When I add delta to this query I get no data:
avg by(consumergroup, topic, cluster)
(delta(kafka_consumergroup_lag{namespace="ns-kafka-int",
consumergroup=~".*$container", cluster="$cluster"}[$__interval]))
So adding delta gives me no result even though when I exclude delta I can
see that the metric has been changing.
The metric is a gaugue.
Question:
Am I doing something wrong?
Can I trust the delta from the Grafana dashboard(someone might know)?
Can I expect a positive or negative value to identify and increase?
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com.
