The @ modifier is no longer experimental. It was made a permanent part of Prometheus in Jan 2022, when the promql-at-modifier was made a no-op. It is no longer listed under feature flags <https://prometheus.io/docs/prometheus/latest/feature_flags/>.
See commit b39f2739e5b01560ad8299d2579f1041a0e9ae5f <https://github.com/prometheus/prometheus/commit/b39f2739e5b01560ad8299d2579f1041a0e9ae5f> which was included in v2.33.0 On Monday 19 August 2024 at 18:34:48 UTC+1 Simon Hardy-Francis wrote: > I think I found one answer to question 2, which is to turn on this via " > --enable-feature=promql-at-modifier" feature [1]. > However, [2] says such " features .. are disabled by default since they > are breaking changes or are considered experimental". So not sure if I want > to use it. > > [1] https://prometheus.io/blog/2021/02/18/introducing-the-@-modifier/ > [2] https://prometheus.io/docs/prometheus/2.45/feature_flags/ > > On Friday, August 16, 2024 at 9:30:27 PM UTC-7 Simon Hardy-Francis wrote: > >> Hello! >> >> I am a relative newbie to Prometheus and promql and have the following >> questions: >> >> Question 1: I tried to find the CPU used by containers in the last 1 >> minute, using 3 different ways via promql, and the ways do not agree with >> each other. >> >> One way is finding the absolute value for " >> container_cpu_usage_seconds_total" before and after and subtracting the >> values. The other way is using "rate" and the last way is using "increase". >> Why do the "rate" and "increase" ways show so much less CPU being used? >> >> Question 2: Let's say I want to find the CPU used for all containers >> active between 1 and 2 hours ago. And to discover how long each container >> is active for the case that e.g. a container only existed for e.g. 10 >> minutes of that 60 minute window. How to create a promql command to do that? >> >> Thanks, >> Simon >> >> P.S.: Here are my commands: >> >> $ cat promql.try-1m.sh >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' 'sort_desc(sum by (instance, namespace, node, pod, container) >> (container_cpu_usage_seconds_total{container!=""}))' > promql.before.txt) & >> sleep 60 >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' 'sort_desc(sum by (instance, namespace, node, pod, container) >> (container_cpu_usage_seconds_total{container!=""}))' > promql.after.txt) & >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' 'sum(rate(container_cpu_usage_seconds_total{container!=""}[1m])) >> by (node, instance, namespace, pod, container)' > promql.rate-1m.txt) & >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' >> 'sum(increase(container_cpu_usage_seconds_total{container!=""}[1m])) by >> (node, instance, namespace, pod, container)' > promql.increase-1m.txt) & >> >> $ ./promql.try-1m.sh >> >> $ ls -al promql.*.txt >> -rw-rw-r-- 1 simon simon 3845801 Aug 16 15:36 promql.before.txt >> -rw-rw-r-- 1 simon simon 3844193 Aug 16 15:37 promql.after.txt >> -rw-rw-r-- 1 simon simon 3565377 Aug 16 15:37 promql.increase-1m.txt >> -rw-rw-r-- 1 simon simon 3591045 Aug 16 15:37 promql.rate-1m.txt >> >> $ cat promql.before.txt | egrep kube-router | head -1 >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 826909.35828385 2024-08-16T15:36:16-07:00 >> >> $ cat promql.after.txt | egrep kube-router | egrep >> "kube-router.*kube-router-kjxnp" >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 826929.923827853 2024-08-16T15:37:16-07:00 >> >> $ perl -e 'printf qq[%f\n], 826929.923827853 - 826909.35828385;' >> 20.565544 >> >> $ cat promql.increase-1m.txt | egrep kube-router | egrep >> "kube-router.*kube-router-kjxnp" >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 5.908468843536704 2024-08-16T15:37:16-07:00 >> >> $ cat promql.rate-1m.txt | egrep kube-router | egrep >> "kube-router.*kube-router-kjxnp" >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 0.09846672408107424 2024-08-16T15:37:16-07:00 >> >> $ perl -e 'printf qq[%f\n], 0.09846672408107424 * 60;' >> 5.908003 >> >> $ perl -e 'printf qq[%f%%\n], (20.565544 - 5.908003) / 5.908003 * 100;' >> 248.096370% >> >> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6cd6b961-893c-4394-90ef-14b0ec7c981cn%40googlegroups.com.
