Maybe worth trying an update to v2.37.6? I saw one memory-related thing in release notes for v2.37.3; it doesn't sound like what you're seeing, but would be easy enough to rule it out.
On Thursday, 13 April 2023 at 15:31:48 UTC+1 Udo Offermann wrote: > It is interesting that Prometheus does its work completely unobtrusively > during data collection - memory consumption remains constant at 4 GB > throughout the day. > Only when I start executing queries, the memory consumption goes through > the roof. > > > > > > Udo Offermann schrieb am Mittwoch, 12. April 2023 um 16:01:56 UTC+2: > >> Hello, >> >> We are running Prometheus 2.37.1 on a VMWare instance with 32GB memory. >> Prometheus keeps crashing with OOM when running some simple queries in >> the web frontend. >> >> Here some key values from the TSDB status page: >> Number of Series 49245 >> Number of Chunks 143382 >> Number of Label Pairs 3585 >> >> What could be the reason for Prometheus crashing all the time? >> >> For example: I just ran java_lang_Memory_HeapMemoryUsage_used a few >> times and the memory usage went up to 22GB! After executing node_load1, >> prometheus allocates nearly 29GB. Finally Prometheus crashed after I >> increased the range two 1 week. >> >> Any help is welcome >> Udo >> >> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/60a2835a-992e-488d-8602-75251b5ae438n%40googlegroups.com.
