Ok, gotcha thanks. Part of the reason I was confused is because Monitoring in gcp has a metric called "container_memory_used_bytes" where you can filter on evictable and non-evictable. If filtering on non-evictable the value seems to correlate with "container_memory_working_set_bytes".
Just adding that here in case someone else stumbles upon this conversation. On Thu, Feb 9, 2023 at 11:03 AM Ben Kochie <[email protected]> wrote: > Kubernetes will not OOM a pod for page cache use. The > container_memory_used_bytes includes page cache and is not what you want to > compare. What you want to do is look at is container_memory_rss or > container_memory_working_set_bytes. > > On Thu, Feb 9, 2023 at 10:56 AM 'Björn Johansson' via Prometheus Users < > [email protected]> wrote: > >> I'm struggling to understand how prometheus uses memory. I've done some >> reading and I find pieces of information here and there but I don't get the >> full picture. >> >> Some context first. >> We are running prometheus in kubernetes. Not the operator, just vanilla >> prometheus. >> We are monitoring memory usage with the >> "container_memory_limit_utilization" and "container_memory_used_bytes" >> metrics. >> >> I've seen in some previous conversation that these metrics are not ideal >> for monitoring usage since they include the page cache as well. This is >> where I start getting confused. >> Kubernetes doesn't make any distinction if memory is the page cache or >> the actual memory used by prometheus. The pod will OOM and be evicted once >> it hits the limit. >> Will page cache be released when needed and before the limit is hit? >> >> Can you please help me understand how this works. >> How should I monitor prometheus so that I can be sure it has the memory >> it needs? >> >> Thanks >> Björn >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/6428d3f7-2fff-4783-9e4d-13f1a5e442c8n%40googlegroups.com >> <https://groups.google.com/d/msgid/prometheus-users/6428d3f7-2fff-4783-9e4d-13f1a5e442c8n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- Björn Johansson Software Developer Cell: +46 72 052 58 49 www.storytel.com Storytel Sweden AB Box 24167 104 51 Stockholm Visit address: Västra Torggatan 19, 65224 Karlstad -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAFLnuWELTP9qxc-99HguTtsLawkTJL%3DTOrOxL14MO5XwpfqQFw%40mail.gmail.com.
