I can't speak for the project, but my guess would be it's out of scope.
There are many existing integrations which use the existing webhook
mechanism:
https://prometheus.io/docs/operating/integrations/#alertmanager-webhook-receiver
Doing it externally allows the request to be enriched with other data which
wouldn't be available in a template, and to use a variety of outbound
authentication mechanisms which Alertmanager itself doesn't implement, or
even non-HTTP protocols like STOMP.
In any case, text templating is a nasty and insecure way to create JSON or
XML payloads.
> it is possible to converse the receivable JSON on the hop and just send
notifications via curl method
You'd first have to *receive* the webhook payload from Alertmanager, which
involves plugging your curl script into a webserver somehow. You could
probably write one as a CGI script under Apache, for example.
But why would you do that, when you have an existing tested and working
solution?
On Monday, 12 June 2023 at 08:13:50 UTC+1 Near N wrote:
> Thanks for reply Brian. I do realize, that Google Chat API requires a
> particular JSON format, but was assuming Alertmanager is capable of doing
> it by using particular template, which is wrong. Now the questions is
> whether it is worth creating a PR to create such a feature or it is out of
> Alertmanager's standard scope?
>
> Another question is perhaps it is possible to converse the receivable JSON
> on the hop and just send notifications via curl method? I mean if the
> endpoint contains the following alert "
> [{"annotations":{"server":"main","number_of_nodes":"1"},"endsAt":"2023-06-12T07:02:00.233Z","fingerprint":"b21bf244f87cd47f","receivers":[{"name":"my_alerts"}],"startsAt":"2023-06-12T06:44:00.233Z","status":{"inhibitedBy":[],"silencedBy":[],"state":"active"},"updatedAt":"2023-06-12T06:58:00.424Z","generatorURL":"
> http://app1:8880/vmalert/alert?group_id=71689142540802366","labels":{"alertgroup":"INFRA","alertname":"The
>
> number of available Cassandra nodes has
> changed","server":"main","severity":"Critical"}}]", would it be possible to
> retrieve this json and post it via curl? This, I assume, wouldn't solve my
> problem, cause I believe Google API expects something peculiar in terms of
> the message it receives.
>
> понедельник, 12 июня 2023 г. в 13:04:17 UTC+6, Brian Candler:
>
>> > Invalid JSON payload received.
>>
>> Alertmanager's webhook sends a JSON message with a *fixed* payload
>> format. See
>> https://prometheus.io/docs/alerting/latest/configuration/#webhook_config
>> *"The Alertmanager will send HTTP POST requests in the following JSON
>> format to the configured endpoint"*
>>
>> You can't override it with a template. The Alertmanager JSON format is
>> different to what Google Chat requires, and therefore you can't send it
>> directly.
>>
>> That's why you need a separate piece of software to do the conversion
>> (Calert in your case) - it receives the Alertmanager webhook JSON payload,
>> builds a new payload for the target, and then sends it there.
>>
>> On Monday, 12 June 2023 at 06:59:14 UTC+1 Near N wrote:
>>
>>> Good day. I was wondering if there is a way to configure alertmanager to
>>> send notifications to google chat via a webhook.
>>>
>>> The current setup that I use consists of vmalert (loaded with set of
>>> promQL compatible rules) >> Alertmanager (to group them up) and Calert (to
>>> send them to google chat). Despite the flowchart is working fine, I found
>>> it difficult to amend proposed message template, so decided to simplify it
>>> by configuring alertmanager to send the alerts. It dawns on me that it can
>>> operate, but I seem to have failed to fine-tune it. Any help would be
>>> appreciated. The alertmanager.yml is below:
>>>
>>> global:
>>> http_config:
>>> proxy_url: '{{ alertmanager_proxy_url }}'
>>>
>>> route:
>>> receiver: 'my_alerts'
>>> group_wait: 30s
>>> group_interval: 60s
>>> repeat_interval: 15m
>>> group_by: ['room', 'alertName']
>>>
>>> receivers:
>>> - name: 'my_alerts'
>>> webhook_configs:
>>> - url: '{{ alertmanager_chat_webhook }}'
>>> send_resolved: true
>>>
>>> templates:
>>> - '{{ alertmanager_working_directory }}/google-chat.tmpl'
>>>
>>>
>>> And here is the error of alertmanager trying to send an alert:
>>>
>>> Jun 12 05:26:00 app1-cp alertmanager[140337]:
>>> ts=2023-06-12T05:26:00.599Z caller=dispatch.go:352 level=error
>>> component=dispatcher msg="Notify for alerts failed" num_alerts=3
>>> err="my_alerts/webhook[0]: notify retry canceled due to unrecoverable error
>>> after 1 attempts: unexpected status code 400:
>>> https://chat.googleapis.com/v1/spaces/webook_token: {\n \"error\": {\n
>>> \"code\": 400,\n \"message\": \"Invalid JSON payload received.
>>> Unknown name \\\"receiver\\\" at 'message': Cannot find field.\\nInvalid
>>> JSON payload received. Unknown name \\\"status\\\" at 'message': Cannot
>>> find field.\\nInvalid JSON payload received. Unknown name \\\"alerts\\\" at
>>> 'message': Cannot find field.\\nInvalid JSON payload received. Unknown name
>>> \\\"groupLabels\\\" at 'message': Cannot find field.\\nInvalid JSON payload
>>> received. Unknown name \\\"commonLabels\\\" at 'message': Cannot find
>>> field.\\nInvalid JSON payload received. Unknown name
>>> \\\"commonAnnotations\\\" at 'message': Cannot find field.\\nInvalid JSON
>>> payload received. Unknown name \\\"externalURL\\\" at 'message': Cannot
>>> find field.\\nInvalid JSON payload received. Unknown name \\\"version\\\"
>>> at 'message': Cannot find field.\\nInvalid JSON payload received. Unknown
>>> name \\\"groupKey\\\" at 'message': Cannot find field.\\nInvalid JSON
>>> payload received. Unknown name \\\"truncatedAlerts\\\" at 'message': Cannot
>>> find field.\",\n \"status\": \"INVALID_ARGUMENT\",\n \"details\": [\n
>>> {\n \"@type\": \"type.googleapis.com/google.rpc.BadRequest\
>>> <http://type.googleapis.com/google.rpc.BadRequest%5C>",\n
>>> \"fieldViolations\": [\n {\n \"field\": \"message\",\n
>>> \"description\": \"Invalid JSON payload received. Unknown name
>>> \\\"receiver\\\" at 'message': Cannot find field.\"\n },\n
>>> {\n \"field\": \"message\",\n \"description\":
>>> \"Invalid JSON payload received. Unknown name \\\"status\\\" at 'message':
>>> Cannot find field.\"\n },\n {\n \"field\":
>>> \"message\",\n \"description\": \"Invalid JSON payload received.
>>> Unknown name \\\"alerts\\\" at 'message': Cannot find field.\"\n
>>> },\n {\n \"field\": \"message\",\n
>>> \"description\": \"Invalid JSON payload received. Unknown name
>>> \\\"groupLabels\\\" at 'message': Cannot find field.\"\n },\n
>>> {\n \"field\": \"message\",\n \"description\":
>>> \"Invalid JSON payload received. Unknown name \\\"commonLabels\\\" at
>>> 'message': Cannot find field.\"\n },\n {\n
>>> \"field\": \"message\",\n \"description\": \"Invalid JSON
>>> payload received. Unknown name \\\"commonAnnotations\\\" at 'message':
>>> Cannot find field.\"\n },\n {\n \"field\":
>>> \"message\",\n \"description\": \"Invalid JSON payload received.
>>> Unknown name \\\"externalURL\\\" at 'message': Cannot find field.\"\n
>>> },\n {\n \"field\": \"message\",\n
>>> \"description\": \"Invalid JSON payload received. Unknown name
>>> \\\"version\\\" at 'message': Cannot find field.\"\n },\n
>>> {\n \"field\": \"message\",\n \"description\":
>>> \"Invalid JSON payload received. Unknown name \\\"groupKey\\\" at
>>> 'message': Cannot find field.\"\n },\n {\n
>>> \"field\": \"message\",\n \"description\": \"Invalid JSON
>>> payload received. Unknown name \\\"truncatedAlerts\\\" at 'message': Cannot
>>> find field.\"\n }\n ]\n }\n ]\n }\n}\n"
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/d882d3d8-3cbb-4f26-a832-269f75d0c27an%40googlegroups.com.
