Mark and Cris, On 7/21/2016 12:47 PM, Mark Thomas wrote: > On 21/07/2016 18:17, Berneburg, Cris J. - US wrote: >> From: Mark Thomas [mailto:ma...@apache.org] > > <snip/> > >> Using the Find Leaks button on the Tomcat Manager page on old app >> versions to trigger full garbage collection revealed that the >> memory leak started to happen in the release when Log4J2 was added >> to the app. It did not start happening in the prior release when >> Mybatis was added. > > It isn't essential but that is a good idea to do that to give you an > idea of what you are looking for. > >>> This should help: http://markmail.org/message/fcbvwapt6afyndxn >> >>> 1. Find an app that you can't reload without OOME 2. Get a >>> profiler [...] 3. Reload you app once 4. Use the profiler to look >>> for instances of WebappClassLoader 5. Look for the one with the >>> started attribute == false 6. Trace the GC roots for this >>> instance >> >> >> Used the Java Visual VM to pull a heap dump after the app restart >> and GC. Used Eclipse Memory Analyzer to analyze the heap dumps. >> Found the WebappClassLoader with started == false and used Path to >> GC roots: >> >> <classloader> >> org.apache.logging.log4j.core.jmx.LoggerContextAdminMBean ... >> com.sun.jmx.mbeanserver.StandardMBeanIntrospector >> >> <classloader> org.apache.logging.log4j.core.jmx.LoggerContextAdmin >> ... com.sun.jmx.mbeanserver.StandardMBeanIntrospector >> >> <classloader> org.apache.logging.log4j.core.jmx.StatusLoggerAdmin >> ... com.sun.jmx.mbeanserver.StandardMBeanIntrospector > > The three above look problematic. > >> referent java.util.WeakHashMap$Entry ... java.lang.reflect.Proxy > > That should disappear once the problematic entries have been fixed > >> <classloader> $Proxy3 ... java.lang.reflect.Proxy > > That looks to be related to the previous GC root and should also > disappear once the problematic entries are fixed. > >> referent java.util.WeakHashMap$Entry ... >> org.apache.juli.ClassLoaderLogManager ... (many >> java.util.logging.*) > > Again, those will disappear once the issues are fixed. They are > Tomcat's internal logging > >> I don't see anything pointing back to our code. With no previous >> experience with heap analysis on my part, it looks to me to be due >> to Log4J2. > > Agreed. > >> Changing the args for the call to LogManager.getLogger from >> Class<T> clazz to none made no appreciable difference. > > I wouldn't expect that to make any difference. > >> (Did I somehow set up Log4J2 incorrectly to get it to misbehave?) > > I don't think so. > >> Not sure what else I can do. Report it to the Log4J2 dev group >> perhaps? > > Ask on their users' list. It may be that when you use log4j2 in a > webapp there is some clean-up method you need to call from a > ServletContextListener. >
From the log4j2 web site: https://logging.apache.org/log4j/2.x/manual/webapp.html In servlet spec 3.0 and greater, there appears to be annotation that gets everything running. In servlet spec 2.5, you'll have to add a bunch of stuff to your web.xml. The reference above gives a more detailed explanation and an example for the 2.5 web.xml. I am just starting to get things migrated to log4j2, so I'll know how it goes shortly. Sigh . . . days late, and dollars short. >>> One thing worth noting. Yourkit offers two types of heap dumps. >>> You don't want YourKit's own. There is a JVM bug that prevents >>> some GC roots from being shown in that format and if your leak >>> traces back to one of those you can end up scratching your head >>> for days. >>> >>> Mark >> >> >> Is there a likely chance that the bug interfered with my usage of >> Java Visual VM and Eclipse Memory Analyzer and thus prevented me >> from determining the real problem source? > > No. The bug just hides GC roots. If the only GC roots you see are > for weak references then you might have hit this bug. > > Mark . . . just my two cents /mde/
signature.asc
Description: OpenPGP digital signature
