Thanks, I think I understand what you mean now.
In theory both of the metrics that I’m using to would work just without any manipulation, for example ltmPoolMbrStatusAvailState will return everything I care about and all the labels I need to use to match it’s just once I manipulate the values that it stops showing me everything. So untested but: ‘ltmPoolMbrStatusAvailState or on (instance,ltmPoolMbrStatusNodeName, ltmPoolMbrStatusPoolName,ltmPoolMbrStatusPort) metricfromrecordingrule * 0’ Then the recording rule would be for the initial join: ‘ltmPoolMbrStatusAvailState * on(instance,ltmPoolMbrStatusNodeName, ltmPoolMbrStatusPoolName,ltmPoolMbrStatusPort) group_right(ltmPoolMbrStatusAvailState) ltmPoolMbrStatusEnabledState)‘ I imagine putting that above query where ‘ metricfromrecordingrule’ is in the main query will cause other issues, so am best to use a recording rule to get that data. That also allows me to use metricfromrecordingrule to do my checking for: - enabled but unavailable - disabled but available - disabled and unavailable And then I can do my subtraction based on that. Sorry I’m new to more advanced querying. Is that looking along the right lines? On Tue, 7 Mar 2023 at 17:00, Brian Candler <[email protected]> wrote: > On Tuesday, 7 March 2023 at 16:47:34 UTC Nick Carlton wrote: > > I suppose that if there are for example no nodes that are enabled but not > available, would Prometheus actually know the pools that don’t match this > because they theory they would not be returned at all in the query, > nevermind the count by. > > > That's what I mean by having a stable metric, one which is always present, > that you can combine with "or" or "unless". > > If you just care about the 'instance' label, then the "up" metric can > perform this purpose, which is created by Prometheus for every target it > attempts to scrape. This allows you to alert on the *absence* of a > particular metric from any target, since you know which targets are being > scraped: > https://www.robustperception.io/absent-alerting-for-scraped-metrics/ > > In your case, you need to identify a particular metric which is always > present for each load balancer pool (or whatever it is you're monitoring) > regardless of its state. > > But no, sorry, I am not going to write this for you :-) It would involve > digging down too far into the details of these F5 metrics than I care to. > Good luck! > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Prometheus Users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/prometheus-users/qlHK7r1hQVs/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/4b1199ef-4dc5-4479-b04d-28d372ea0dd7n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/4b1199ef-4dc5-4479-b04d-28d372ea0dd7n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAC4WY5_FuWLvS1zi7wV%3DwdT0KXKz6PWNEocDJLa2VpOc2BHvnA%40mail.gmail.com.
