For this, container_ metrics are what they want. Summed by pod. The problem is they are using a regexp to match the pod name, rather than using a label like "job", "deployment" or some other higher-level matching. The regexp is too fragile.
Without knowing what other labels they have available, it's hard to say. They may also need to improve their relabel configs to better map Kubernetes metadata onto the container metrics. On Sat, Mar 4, 2023 at 10:01 AM Brian Candler <[email protected]> wrote: > Your query is asking for metrics at the *pod* level - > *container_*memory_working_set_bytes > - and that's what you're getting. > > If you want to see which pods were started by specific deployments, you'll > need to find a metric > <https://github.com/kubernetes/kube-state-metrics/blob/main/docs/pod-metrics.md> > which gives this relationship. What does kube_pod_info show? (In > particular, the labels created_by_kind, created_by_name) > > Then you should be able to use a one-to-many relationship in your query to > filter the pods. > > https://prometheus.io/docs/prometheus/latest/querying/operators/#many-to-one-and-one-to-many-vector-matches > https://www.robustperception.io/how-to-have-labels-for-machine-roles > https://www.robustperception.io/exposing-the-software-version-to-prometheus > > On Saturday, 4 March 2023 at 07:58:09 UTC Moksh wrote: > >> Hi Team, >> >> I've a problem while fetching the metrics for the dedicated deployment. I >> don't see any deployment related metrics in Prometheus to get the >> deployment specific resource usage with kube-state-metrics. >> >> For Example: When I use the regex for a pod_name on container specific >> metric its populating all #thanos related metrics including sub components >> like; thanos-receiver, thanos-frontend, etc. My current PromQL is as >> follows. It should show thanos deployment metrics only, please look into >> this issue and help me out! >> >> sum(container_memory_working_set_bytes{container_name!="", >> container_name!="POD", >> pod_name=~"thanos-(([a-z0-9]*)|([a-z0-9]*-[a-z0-9]*)|[0-9])", >> namespace="$namespace", k8s_cluster="$cluster_name"}) by (id,pod_name) >> >> Current output: >> {pod_name="thanos-6c4d9c7bf7-ggnem"} {pod_name="thanos-6c4d9c7bf7-h6tld"} >> {pod_name="thanos-receiver-0"} {pod_name="thanos-receiver-1"} >> >> Expected output: >> {pod_name="thanos-6c4d9c7bf7-ggnem"} {pod_name="thanos-6c4d9c7bf7-h6tld"} >> >> Looking forward to hearing from you, thanks! >> >> Best Regards, >> Moksha >> > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/bc09a69a-3b97-4b20-8b34-21cb0992ed87n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/bc09a69a-3b97-4b20-8b34-21cb0992ed87n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmo-1jGtjWqVY3pWfhuaLKtM%2BU3qeo82hL8u5q8OEEDF5A%40mail.gmail.com.
