Are there any opinions about whether or not to extend NMT across the JDK?
This blocks https://bugs.openjdk.org/browse/JDK-8296360
<https://bugs.openjdk.org/browse/JDK-8296360>, and I had a PR prepared
as https://github.com/openjdk/jdk/pull/10988
<https://github.com/openjdk/jdk/pull/10988>. Originally I was hoping to
get this into JDK 20, but I don't think that is realistic anymore. I am
fine with postponing my work in favor of a baseline discussion, but so
far there is very little discussion about this topic.
How should I proceed?
Thanks, Thomas
On Wed, Nov 9, 2022 at 8:12 AM Thomas Stüfe <thomas.stu...@gmail.com
<mailto:thomas.stu...@gmail.com>> wrote:
Hi Alan,
(replaced hotspot-runtime-dev with hotspot-dev, since its more of a
general topic)
thank you for your time!
I am very happy to talk this through. I think native memory
observability in the JDK (and customer code!) is sorely lacking.
Witness the countless "where did my native memory go" blog articles.
At SAP we have been struggling with this topic for a long time and
have come up with a mixture of solutions. The aforementioned tracker
was one, which extended our version of NMT across the JDK. Our
SapMachine MallocTracer, which allows us to trace uninstrumented
customer code, another. We even experimented with exchanging the
allocator (using jemalloc) to gain insights. But that is a whole
different topic with deep logistical implications, I don't want to
touch it here. Exchanging the allocator does not help to observe
virtual memory or the brk segment, of course.
And to make the picture complete, another insight we currently lack
is the implicit allocator overhead, which can be very significant
and is hidden by the libc. We also have observability for that in
the SapMachine, and I miss it in OpenJDK.
As you noticed, my original intent was just to instrument Zlib and
possibly improve tracking for DBBs. Although, thinking beyond that,
another attractive instrumentation target would be mapped NIO
buffers at least.
So I think native memory observability is important. Arguably we
could even extend observability to cover other OS resources, e.g.
file handles. If we shift code around, to java/Panama: data that
move the java heap does not need to be tracked, but other memory
will always come from one of the basic system APIs, regardless of
who allocates it and where in the stack allocation happens. Be it
native JDK code, Panama, or even customer JNI code.
If we agree on the importance of native memory observability, then I
believe NMT is the right tool for it. It is a good tool. The
machinery is already there. It covers both C-heap and virtual memory
APIs, as well as thread stacks, and could easily be extended to
cover sbrk if needed. And I assume that whatever shape OpenJDK takes
on in the future, there always will be a libjvm.so at its core, so
we will always have it. But even if not, NMT could be separated from
libjvm.so quite easily, since it has no deep ties with the JVM.
About coupling JVM with outside code: We don't have to directly link
against libjvm.so. We can keep things loose if the intent is to be
runnable without a JVM, or be JVM-version-agnostic. That could take
the form of a function-pointer interface like JVMTI. Or outside code
could dynamically dlsym the JVM allocation hooks. In any case
gracefully falling back to system allocation routines when necessary.
And I agree, polluting the NMT tag space with outside meaning is
ugly. I only did it because I planned to go no further than
instrumenting Zlib and possibly DBBs. But if we take this further,
my preferred solution would be a reserved tag range or -ranges for
outside use, whose inner meaning would be opaque to the JVM. Kind of
like SIGRTMIN+SIGRTMAX. Then, outside code could register tags and
their meta information with the JVM, or we find a different way to
convey the tag meaning to NMT (config files, or callbacks). That
could even be opened up for customer use.
This also touches on another question, that of NMT tag space. NMT
tags are very useful since they allow cheap tracking without
capturing call stacks. However, tags are underused and show growing
pains since they are too one-dimensional and restrictive. We had
competing interests in the past about tag granularity. It is all
over the place. We have coarse-grained tags like "mtThread", and
very fine-grained ones like "mtObjectMonitor". There are several
ways we could improve, e.g., by making them combinable like UL does,
or allowing for a hierarchy of them - either a hard-wired limited
one like "domain"+"tag", or an unlimited tree-like one. Technically
interesting since whatever the new encoding is, they still must fit
into a malloc header. I opened
https://bugs.openjdk.org/browse/JDK-8281819
<https://bugs.openjdk.org/browse/JDK-8281819> to track ideas like these.
Instrumenting Panama allocations, including the ability to tag
allocations, would be a very good idea. For instance, if we ever
remove the native Zlib layer and convert it to java using Panama, we
can do the same with Panama I do now natively - use the Zlib zalloc
interface to hook in JVM memory allocation functions. The result
could be completely identical, and the end user looking at the NMT
output need never know that anything changed.
And that goes for all instrumentation - if today we add it to JNI
code, and that code gets removed tomorrow, we can add it to Panama
code too. Unless data structures move to the heap, in which case
there is no need to track them.
You mentioned that NMT was more of an in-house support tool. Our
experience is different. Even though it was positioned as a tool for
JVM developers, and we never cared for the backward compatibility or
consistency, it gets used a *lot* by our customers. We have to
explain its output frequently. Also, many blog articles exist
documenting its use. So, maybe it would be okay to elevate it to a
user-facing tool since it seems to occupy that role anyway. We may
also open up consumption of NMT results via java APIs, or expose its
results via MXBeans.
If this is to be a JEP, okay, but I'm afraid it would stall things a
bit. I am interested in getting a simpler and quicker solution for
older support releases at least, possibly based on my PR. I know
that would be unconventional though.
Thank you,
Thomas
On Sun, Nov 6, 2022 at 9:31 AM Alan Bateman <alan.bate...@oracle.com
<mailto:alan.bate...@oracle.com>> wrote:
On 04/11/2022 16:54, Thomas Stüfe wrote:
> Hi all,
>
> I am currently working on
https://bugs.openjdk.org/browse/JDK-8296360
<https://bugs.openjdk.org/browse/JDK-8296360>;
> I was preparing the final PR [1], but then Alan did ask me to
discuss
> this on core-libs first.
>
> Backstory:
>
> NMT tracks hotspot native allocations but does not cover the JDK
> libraries (small exception: Unsafe.AllocateMemory). However, the
> native memory footprint of JDK libraries can be significant.
We have
> no in-VM tracker for these and need tools like valgrind or our
> SapMachine MallocTracer [2] to observe them.
Thanks for starting a discussion on this as this is a topic that
requires agreement from several areas. If this is the start of
something
bigger, where you want to have all allocation sites in the
libraries
using NMT, then I think it needs a write-up, maybe a JEP.
For starters, I think it needs some agreement on using NMT for
memory
allocated outside of libjvm. You mentioned Unsafe as an
exception but
that is implemented in the VM so you get tracking for free,
albeit I
think all allocations are in the "mtOther" category.
A general concern is that it creates more coupling between the
VM code
and the libraries code. As you probably know, we've removed most
of the
dependences on JVM_* functions from non-core areas over many
years. So I
think that needs consideration as I assume we don't want
memory/allocation.hpp declaring a dozen catagories for
allocations done
in say java.desktop module for example. Maybe your proposal will be
strictly limited to java.base but even then, do we really want
the VM
even knowing about categories that are specific to zip
compression or
decompression?
There are probably longer term trends that should be part of the
discussion too. One general trend is that "run time" is becoming
more
and more a hybrid of code in libvm and the Java libraries. Lambdas,
module system, virtual threads implementations are a few
examples in the
last few release. This comes with many "Java on Java" challenges,
including serviceability where users of the platform will expect
tools
to just work and won't care where the code is. NMT is probably
more for
support teams and not something that most developers will ever
use but I
think is part of the challenge of having serviceability
solutions "just
work".
In addition to having more of the Java runtime written in Java,
there
will likely be less JNI code in the future. It's very possible
that the
JNI code (including the JNI methods in libzip) will be replaced
with
code that uses Panama memory and linker APIs once they are become
permanent. The effect of that would to have a lot of the memory
allocations be tracked in the mtOther category again. Maybe
integration
with memory tracking should be looked at in conjunction with
these APIs
and this migration. I could imagine the proposed "Arena" API
(MemorySession in Java 19) having some integration with NMT and
it might
be interesting to look into that.
So yes, this topic does need broader discussion and it might be
a bit
premature to start with a PR for libzip without talking about
the bigger
picture first.
-Alan
