It is not clear what are you trying to accomplish with on(namespace, pod)
group_left(label_seldon_deployment_id) clauses. It seems like all three of
your metrics are expected to have all three of these labels.
Nevertheless, if all three sums produce output, and all three outputs
contain same labels values you should get some result.
четверг, 24 августа 2023 г. в 20:17:59 UTC+3, kumar yele:
> hi,
> i have a gke setup with monitoring setup. so i have the requirement of
> using gpu nodes so i have configured the prometheus to get the gpu metrics.
> but when i plot the query using the gpu metrics i got empty query result.
> when i do the queries individually they are showing the data.
> here are the result
> i am using this gpu query ----->DCGM_FI_DEV_GPU_UTIL
> when i combine with my deployment -----> sum by (namespace, pod,
> label_seldon_deployment_id) (DCGM_FI_DEV_GPU_T) * on(namespace, pod)
> group_left(label_seldon_deployment_id) sum by (namespace, pod,
> label_seldon_deployment_id) (DCGM_FI_DEV_GPU_UTIL) * on(namespace, pod)
> group_left(label_seldon_deployment_id)
> kube_pod_labels{label_seldon_deployment_id="llm-testing-1"}
>
> its not giving the result. showing empty result.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/4e9362e7-d7b1-4ad2-a19a-d0dfcad57fc9n%40googlegroups.com.
