On Thu, 9 Feb 2023 18:11:18 GMT, Volker Simonis <simo...@openjdk.org> wrote:
>> Prior to >> [JDK-8239384](https://bugs.openjdk.org/browse/JDK-8239384)/[JDK-8238358](https://bugs.openjdk.org/browse/JDK-8238358) >> LambdaMetaFactory has created VM-anonymous classes which could easily be >> unloaded once they were not referenced any more. Starting with JDK 15 and >> the new "hidden class" based implementation, this is not the case any more, >> because the hidden classes will be strongly tied to their defining class >> loader. If this is the default application class loader, these hidden >> classes can never be unloaded which can easily lead to Metaspace exhaustion >> (see the [test case in the JBS >> issue](https://bugs.openjdk.org/secure/attachment/102601/LambdaClassLeak.java)). >> This is a regression compared to previous JDK versions which some of our >> applications have been affected from when migrating to JDK 17. >> >> The reason why the newly created hidden classes are strongly linked to their >> defining class loader is not clear to me. JDK-8239384 mentions it as an >> "implementation detail": >> >>> *4. the lambda proxy class has the strong relationship with the class >>> loader (that will share the VM metaspace for its defining loader - >>> implementation details)* >> >> From my current understanding the strong link between a hidden class created >> by `LambdaMetaFactory` and its defining class loader is not strictly >> required. In order to prevent potential OOMs and fix the regression compared >> the JDK 14 and earlier I propose to create these hidden classes without the >> `STRONG` option. >> >> I'll be happy to add the test case as JTreg test to this PR if you think >> that would be useful. > > Volker Simonis has updated the pull request incrementally with two additional > commits since the last revision: > > - Remove assertions which insist on Lambda proxy classes being strongly > linked to their class loader > - Removed unused import of STRONG und updated copyright year Hi Volker, just a short feedback since I'm entering vacation mode. > In general it seems to me that creating a new `ClassLoaderData` for each > hidden (or VM anonymous) class is a lot of overhead and was probably only > done as "a hack" to simplify the implementation of class unloading for such > classes. This might have been appropriate in the "early days" when we had > just a few VM anonymous classes. But as the usage of > `LambdaMetafactory.metafactory()` becomes more common it may make sense to > revise the implementation such that for example all hidden classes defined by > a class loader can live in a single `ClassLoaderData` structure. I think this > should be technically possible although it would complicate the unloading > logic (and should therefore be postponed to a later change). Interesting numbers. I underestimated the impact of native allocations. A side note, the amount of waste avoided by JEP 371 depends highly on the actual size of the lambda (precisely, how close a lambda would be to the chunk size). That's pretty much random, but nice to know since it can shift the numbers from almost nil overhead to almost twofold overhead. >I'd argue that Metaspace is more "valuable" than native memory because it is >limited by -XX:MetaspaceSize, whereas native memory is only limited by the >amount of available memory on the system. Also, growing Metaspace will trigger >garbage collections which might be unexpected. I don't follow that logic, for several reasons: - Metaspace is more easily reclaimed, since it returns memory to the OS eagerly (since JEP 387). C-Heap, otoh, sticks to the process unless you (atm) manually trim. Depends on the libc used, but with glibc - arguable the main platform - this is the state. We are working on C-heap autotrim, but it is not completed yet. So while Metaspace is limited, it recovers from spikes more easily. - The metaspace GC threshold is an (impreceise) mechanism to trigger GC based on native resource consumption. That this is not affected by C-heap memory is just a side effect of an imperfect implementation - ideally, all native resources associated with a class loader - be it metaspace or C-heap - together would weight into the decision of when to do a GC. In fact, your numbers may be taken as a hint that maybe CLDs should live in Metaspace instead. --- As I stated before, we do have the ability to prematurely repurpose metaspace before the loader dies via Metaspace::deallocate(). It would be nice if there were a mechanism to trigger this at least from Java. E.g. by making Lambdas closable. ------------- PR: https://git.openjdk.org/jdk/pull/12493
