Or do I need to change it to explicitly say in the crit and warn mail to range over the .Alerts.Firing in order to get the third receiver to work?
receivers: - name: 'email' email_configs: - to: '[email protected]' send_resolved: false headers: subject: 'Critical: {{ .GroupLabels.alertname }} - Instances: {{ range .Alerts.Firing }}{{ .Labels.instance }}, {{ end }}' - name: 'email_warning' email_configs: - to: '[email protected]' send_resolved: false headers: subject: 'Warning: {{ .GroupLabels.alertname }} - Instances: {{ range .Alerts.Firing }}{{ .Labels.instance }}, {{ end }}' - name: 'email_resolved' email_configs: - to: '[email protected]' send_resolved: true headers: subject: 'Resolved: {{ .GroupLabels.alertname }} - Instances: {{ range .Alerts.Resolved }}{{ .Labels.instance }}, {{ end }}' Kolja Krückmann schrieb am Freitag, 16. Juni 2023 um 11:21:59 UTC+2: > I changed the config to this: > > - name: 'email_resolved' > email_configs: > - to: '[email protected]' > send_resolved: true > headers: > subject: 'Resolved: {{ .GroupLabels.alertname }} - Instances: {{ > range .Alerts.Resolved }}{{ .Labels.instance }}, {{ end }}' > > But this is sending absolutly nothing. What did I do wrong? > > Brian Candler schrieb am Freitag, 9. Juni 2023 um 15:57:04 UTC+2: > >> Maybe you want .Alerts.Firing and .Alerts.Resolved. A single alert group >> is likely to contain a mixture of firing alerts and resolved alerts. >> >> Also, note that your "send_resolved: true" receiver doesn't only send >> resolved messages: it will send on firing *and* on resolved. Hence the >> subject "Resolved: ..." is likely to be very misleading. >> >> On Friday, 9 June 2023 at 13:42:13 UTC+1 Kolja Krückmann wrote: >> >>> Got it working like this: >>> >>> receivers: >>> - name: 'email' >>> email_configs: >>> - to: '[email protected]' >>> send_resolved: false >>> headers: >>> subject: 'Critical: {{ .GroupLabels.alertname }} - Instances: {{ >>> range .Alerts }}{{ .Labels.instance }}, {{ end }}' >>> >>> - name: 'email_warning' >>> email_configs: >>> - to: '[email protected]' >>> send_resolved: false >>> headers: >>> subject: 'Warning: {{ .GroupLabels.alertname }} - Instances: {{ >>> range .Alerts }}{{ .Labels.instance }}, {{ end }}' >>> >>> - name: 'email_resolved' >>> email_configs: >>> - to: '[email protected]' >>> send_resolved: true >>> headers: >>> subject: 'Resolved: {{ .GroupLabels.alertname }} - Instances: {{ >>> range .Alerts }}{{ .Labels.instance }}, {{ end }}' >>> >>> Kolja Krückmann schrieb am Donnerstag, 8. Juni 2023 um 15:03:14 UTC+2: >>> >>>> [EDIT] >>>> Also I want to split the firing and resolved alerts. >>>> Kolja Krückmann schrieb am Donnerstag, 8. Juni 2023 um 14:48:40 UTC+2: >>>> >>>>> Hi all, I tried it now for two days straight and I just don't get it >>>>> working. >>>>> >>>>> First of all where do I actually find the expressions I can use to >>>>> define the subject under the header? I am still missing a template I can >>>>> use? I don't really have one in my Prom/Alert-files. >>>>> Second of all my current receivers are configured like so: >>>>> >>>>> receivers: >>>>> - name: 'email' >>>>> email_configs: >>>>> - to: '[email protected]' >>>>> send_resolved: true >>>>> headers: >>>>> Subject: "{{ .Alerts.alertname }} - {{ range .Alerts }} {{ >>>>> .Alerts.instance }} {{ end }}" >>>>> >>>>> Like Brian said, I want to iterate in the second half of that >>>>> "statement" (I don't actually know how these {{ .xyz }} are called) >>>>> through >>>>> all instances and the first one should just say which alert is now firing. >>>>> I also tried it as followed: >>>>> >>>>> Subject: "{{ .GroupLabels.alertname }} - {{ range .Alerts }} {{ >>>>> .Alerts.instance }} {{ end }}" >>>>> and >>>>> Subject: "{{ .Labels.alertname }} - {{ range .Alerts }} {{ >>>>> .Alerts.instance }} {{ end }}" >>>>> >>>>> but nothing is really working. >>>>> >>>>> All I really want is the Subject like so e.x.: >>>>> InstanceDown - SVR-DS01 SVR-DC11 >>>>> ^^^^^^^^ ^^^^^^^^ ^^^^^^^^ >>>>> Alertname - Instance1 Instance2 (if it is also possible to >>>>> seperate the instances via comma would be perfect) >>>>> Brian Candler schrieb am Dienstag, 6. Juni 2023 um 13:16:53 UTC+2: >>>>> >>>>>> As its name implies, "CommonLabels" contains only those labels which >>>>>> are common to all alerts in the group. If there are multiple instances, >>>>>> then the "instance" label is not common to all alerts, so won't be in >>>>>> this >>>>>> object. >>>>>> >>>>>> You want to iterate over "Alerts" (or "Alerts.Firing") and look at >>>>>> the Labels.instance within each alert. You will find an example showing >>>>>> how >>>>>> to do that here: >>>>>> >>>>>> https://prometheus.io/docs/alerting/latest/notification_examples/#ranging-over-all-received-alerts >>>>>> >>>>>> Additional references: >>>>>> https://prometheus.io/docs/alerting/latest/notifications/#data >>>>>> >>>>>> https://prometheus.io/docs/prometheus/latest/configuration/template_reference/ >>>>>> >>>>>> On Tuesday, 6 June 2023 at 12:03:03 UTC+1 Kolja Krückmann wrote: >>>>>> >>>>>>> Currently my alertmanager.yml looks like this: >>>>>>> >>>>>>> receivers: >>>>>>> - name: 'email' >>>>>>> email_configs: >>>>>>> - to: '[email protected]' >>>>>>> send_resolved: true >>>>>>> headers: >>>>>>> Subject: '{{ .CommonLabels.alertname }} - {{ >>>>>>> .CommonLabels.instance }}' >>>>>>> >>>>>>> Unfortunately the subject in the mail now only displays both labels >>>>>>> if only there is an alert for a single instance (like High CPU Usage on >>>>>>> core 0,1,2,3 on one instance (see screenshot attatched)) As soon as the >>>>>>> alert is for more then just one instance the instance lable is >>>>>>> completly >>>>>>> missing... Why is it missing? And how can I change it to display all >>>>>>> instances affected by that alert. >>>>>>> >>>>>>> Kind regards. >>>>>>> Kolja >>>>>>> >>>>>>> [image: mail.png] >>>>>>> >>>>>>> Kolja Krückmann schrieb am Montag, 5. Juni 2023 um 09:21:22 UTC+2: >>>>>>> >>>>>>> Hi y'all >>>>>>> I'm trying to change the way the mail from my alertmanger looks and >>>>>>> behaves. >>>>>>> My goal is to have the subject list the alertname and the targets >>>>>>> detected by that alert. (I need it like that for future todo's) >>>>>>> My first problem is, i dont have any templates in my files. Do I >>>>>>> need to clone one from git? How does the prom know, I have a template? >>>>>>> The second one would be to change the subject in order for my >>>>>>> requirements to fit. >>>>>>> >>>>>>> Kind regards, >>>>>>> Kolja >>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/0591b750-b43b-4233-abe1-bbe9b07382a1n%40googlegroups.com.
