*+serviceability-dev* Firstly, thank you both for your time and work in this space. Apologies if this should be a separate thread, but the new title “Extend Native Memory Tracking over the JDK” aligns directly with some work I’ve been investigating, and I hope my feedback will be helpful for prioritization of zlib observability as well as the way users think about native memory tracking in general.
Observability of native memory in the JVM is critically important, and becomes even more valuable as the industry shifts to more and smaller services deployed in right-sized container environments like kubernetes. Each new JDK release (major and hotfix) offers dramatic improvements, often based on some form of trade-off. To be clear, I cannot overstate how impressed I am with quality and velocity of improvement! However, these trade-offs impact the way that memory is used, and it’s a difficult balance to ensure containers use the correct amount of memory without being wasteful (over-provisioned) or oomkilled (under-provisioned). In production, I have thousands of JVMs running with native memory tracking summary enabled. Real-time monitoring of the output is painful and inefficient. Currently the only supported option I’m aware of is shelling out to create a new jcmd process and parsing the NMT summary text output periodically. In older releases, it was possible to bypass the jcmd process by self-attaching, but that was limited in jdk9 by JDK-8178380 <https://bugs.openjdk.org/browse/JDK-8178380>, and still required the caller to parse human-readable strings. In fact, attachment issues in some JDK/environment combinations make automated attachment *dangerous* in a way that has crashed the JVM — that may be a story for another day, but my point is that simple, efficient NMT data collection would go a very long way. Many modern observability tools, especially those used in container deployments, operate by reading data from within the jvm process, and relaying it to a storage system (Prometheus may be the most ubiquitous example). For my use-case, I’d love to have a simple API I could invoke from java code to access structured native-memory-tracking data, similar in a way to MemoryPoolMXBean for heap pools (although JMX isn’t necessary for me, it aligns with other observability APIs in the JDK). Additionally, I’d like to provide JFR events to periodically record native memory tracking metadata when enabled for better out-of-the-box experience with JMC. I’ve begun investigating some options for JDK-8182634 <https://bugs.openjdk.org/browse/JDK-8182634>, but would appreciate feedback before I propose any sort of code change. Thank you all for beginning this discussion, I’m eager to see the ways the JDK continues to improve upon observability features, and do what small part I can to help! Carter Kozak
