I did resolve this by going back to relabel_configs with some other examples I found, so the exemption is working. I do hear you about the high cpu style alerting but our team seems to want it — will dig in some more.
The netbox http sd we’re using is: https://github.com/FlxPeters/netbox-plugin-prometheus-sd Thanks for your help! Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: [email protected] <[email protected]> on behalf of Brian Candler <[email protected]> Sent: Sunday, July 3, 2022 5:16 AM To: Prometheus Users <[email protected]> Subject: Re: [prometheus-users] Service Discovery, Relabeling for Alert Exemptions Out of interest, what's the netbox service discovery plugin you're using - can you give a link please? > when i try to use the separators to tell it there's a comma the yaml stops > parsing Then it must be a syntax error in your yaml. There are cases when strings *must* be quoted, and since comma has significance within YAML itself, this is probably one of those cases. You'd need: separator: ',' But it doesn't matter because this setting won't do anything anyway here. It's used when concatenating the values of multiple "source_labels:", but you have listed only one. > Unfortunately, the netbox tag list doesn't contain leading and trailing > commas in the list, so you can't use the match of tags!~".*,highcpuexempt,.*" > to get more precise matching. At least not without a more fancy regexp. I think either of these should work: tags!~"(.*,|)highcpuexempt(.*,|)" tags!~"(.*,)?highcpuexempt(.*,)?" > This style of "My CPU is too high" is toil prone. They generate false > positives leading to alert fatigue. I strongly agree. Another useful reference: https://docs.google.com/document/d/199PqyG3UsyXlwieHaqbGiWVa8eMWi8zzAn0YfcApr8Q/ -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]<mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/05de56a9-6779-438f-9c8d-9bdd0cf94337n%40googlegroups.com<https://groups.google.com/d/msgid/prometheus-users/05de56a9-6779-438f-9c8d-9bdd0cf94337n%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/DM5PR07MB4133FB76A7087717F1DD6F2AF1BF9%40DM5PR07MB4133.namprd07.prod.outlook.com.
