Repeating what I said before, I would do it by adjusting the alerting expr
to return the actual amount of free space in bytes:
- alert: LowDiskSpace
expr: *windows_logical_disk_free_bytes and*
((windows_logical_disk_free_bytes{volume="C:"} /
windows_logical_disk_size_bytes) * 100) <= 10
for: 1m
labels:
severity: warning
annotations:
summary: "10% C-Disk Space"
description: "Das C-Volumen des Hosts {{ $labels.instance }} ist noch
10% frei *({{$value | humanize}})*."
- alert: LowDiskSpace
expr: *windows_logical_disk_free_bytes and*
((windows_logical_disk_free_bytes{volume="C:"} /
windows_logical_disk_size_bytes) * 100) <= 5
for: 1m
labels:
severity: critical
annotations:
summary: "5% C-Disk Space"
description: "Das C-Volumen des Hosts {{ $labels.instance }} ist noch
5% frei *({{$value | humanize}})*."
Since the percentage is never returned as part of the query, I would also
simplify the expression to remove the factor of 100.
expr: *windows_logical_disk_free_bytes and*
(windows_logical_disk_free_bytes{volume="C:"}
/ windows_logical_disk_size_bytes) < 0.1
Note: you can simply enter the entire "expr" into the PromQL browser in the
Prometheus web interface to test it. If it returns one or more values, that
means an alert condition is active. You'll also be able to see the values
and labels returned by the alerting expression. I have tested with the
following expression:
node_filesystem_avail_bytes and node_filesystem_avail_bytes /
node_filesystem_size_bytes < 0.7
The "and" operator in PromQL is documented
here:
https://prometheus.io/docs/prometheus/latest/querying/operators/#logical-set-binary-operators
On Wednesday, 10 May 2023 at 12:51:48 UTC+1 Kolja Krückmann wrote:
> Thanks alot!
>
> My question now is, that my current expressions only returns the
> percentage as value - can I just add another expression for that specific
> alert?
> Currentliy my alerts for diskSpace (one at 10% as warning and one at 5% as
> crit):
>
> - alert: LowDiskSpace
> expr: ((windows_logical_disk_free_bytes{volume="C:"} /
> windows_logical_disk_size_bytes) * 100) <= 10
> for: 1m
> labels:
> severity: warning
> annotations:
> summary: "10% C-Disk Space"
> description: "Das C-Volumen des Hosts {{ $labels.instance }} ist
> noch 10% frei *({{$value | humanize}})*."
>
> - alert: LowDiskSpace
> expr: ((windows_logical_disk_free_bytes{volume="C:"} /
> windows_logical_disk_size_bytes) * 100) <= 5
> for: 1m
> labels:
> severity: critical
> annotations:
> summary: "5% C-Disk Space"
> description: "Das C-Volumen des Hosts {{ $labels.instance }} ist
> noch 5% frei *({{$value | humanize}})*."
>
> I want the values (bold above) to get returned as gigabyte values -> so
> can I just add an expression which calculates the free space?
>
> Brian Candler schrieb am Mittwoch, 10. Mai 2023 um 13:36:13 UTC+2:
>
>> The "result" of the expression is available as {{ $value }}, and there
>> are functions to convert this into a more human-readable value. See
>> https://prometheus.io/docs/prometheus/latest/configuration/template_reference/
>>
>> Examples:
>>
>> expr: windows_logical_disk_free_bytes{volume="C:"} /
>> windows_logical_disk_size_bytes
>> annotations:
>> description: "Low free disk space: {{ $value | humanizePercentage }}"
>>
>> expr: windows_logical_disk_free_bytes{volume="C:"} < 1000000000
>> annotations:
>> description: "Low free disk space: {{ $value | humanize }}" # or
>> humanize1024: depends if you want Gigabytes or Gibibytes.
>> https://en.wikipedia.org/wiki/Gigabyte
>>
>> If you want to do the threshold based on percent, but report the absolute
>> value, I would use something like this (untested):
>>
>> expr: windows_logical_disk_free_bytes and
>> (windows_logical_disk_free_bytes{volume="C:"} /
>> windows_logical_disk_size_bytes < 0.1)
>> annotations:
>> description: "Low free disk space: {{ $value | humanize }}"
>>
>> I believe it's also possible to embed a completely separate query in a
>> template (to look up a separate value to include in the annotations), but
>> I've never done it, and can't find any examples.
>>
>> Aside: I find these sort of static alerts annoying. Sometimes a
>> filesystem has 8% disk free space and that's a good and normal situation
>> for it to be in. Therefore, either you're lost in a sea of unimportant
>> repeating alerts, or you're jumping through hoops for setting separate
>> static thresholds per filesystem.
>> https://groups.google.com/g/prometheus-users/c/wHLxUPtrb-A/m/idIcdJIrBgAJ
>>
>> Another approach you could consider:
>> https://groups.google.com/g/prometheus-users/c/0ncUqLm0LhU/m/mAiwaADXAgAJ
>> This looks at how quickly the filesystem is filling up, and tells you how
>> long before it expects to be full.
>>
>> On Wednesday, 10 May 2023 at 09:23:27 UTC+1 Kolja Krückmann wrote:
>>
>>> Small correction here:
>>>
>>> I want to have the expression
>>> "windows_logical_disk_free_bytes{volume="C:"}/1000/1000/1000" (if this is
>>> the actual GB of free disk space (or do I need to device by 1024?)) in my
>>> alerting mail. And not as above the percentage.
>>>
>>> Kolja Krückmann schrieb am Mittwoch, 10. Mai 2023 um 10:20:53 UTC+2:
>>>
>>>> Hi y'all
>>>>
>>>> I'm looking for a possibility to add the "result" of an expression to
>>>> the alerting description.
>>>> My expression is to alert when the c:\ Drive is below 10%. Now I want
>>>> to add the actual value of the expression:
>>>> ((windows_logical_disk_free_bytes{volume="C:"} /
>>>> windows_logical_disk_size_bytes) * 100) <= 10
>>>> in the alerting mail. Is this somehow possible so that the free size is
>>>> within the mail?
>>>>
>>>> Kind regards
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/8713feb0-c558-4f8a-946f-7e103c0c20e0n%40googlegroups.com.
