On 21/07/2016 18:17, Berneburg, Cris J. - US wrote: > From: Mark Thomas [mailto:ma...@apache.org]
<snip/> > Using the Find Leaks button on the Tomcat Manager page on old app versions to > trigger full garbage collection revealed that the memory leak started to > happen in the release when Log4J2 was added to the app. It did not start > happening in the prior release when Mybatis was added. It isn't essential but that is a good idea to do that to give you an idea of what you are looking for. >> This should help: >> http://markmail.org/message/fcbvwapt6afyndxn > >> 1. Find an app that you can't reload without OOME >> 2. Get a profiler [...] >> 3. Reload you app once >> 4. Use the profiler to look for instances of WebappClassLoader >> 5. Look for the one with the started attribute == false >> 6. Trace the GC roots for this instance > > > Used the Java Visual VM to pull a heap dump after the app restart and GC. > Used Eclipse Memory Analyzer to analyze the heap dumps. Found the > WebappClassLoader with started == false and used Path to GC roots: > > <classloader> org.apache.logging.log4j.core.jmx.LoggerContextAdminMBean ... > com.sun.jmx.mbeanserver.StandardMBeanIntrospector > > <classloader> org.apache.logging.log4j.core.jmx.LoggerContextAdmin ... > com.sun.jmx.mbeanserver.StandardMBeanIntrospector > > <classloader> org.apache.logging.log4j.core.jmx.StatusLoggerAdmin ... > com.sun.jmx.mbeanserver.StandardMBeanIntrospector The three above look problematic. > referent java.util.WeakHashMap$Entry ... java.lang.reflect.Proxy That should disappear once the problematic entries have been fixed > <classloader> $Proxy3 ... java.lang.reflect.Proxy That looks to be related to the previous GC root and should also disappear once the problematic entries are fixed. > referent java.util.WeakHashMap$Entry ... > org.apache.juli.ClassLoaderLogManager ... (many java.util.logging.*) Again, those will disappear once the issues are fixed. They are Tomcat's internal logging > I don't see anything pointing back to our code. With no previous experience > with heap analysis on my part, it looks to me to be due to Log4J2. Agreed. > Changing the args for the call to LogManager.getLogger from Class<T> clazz > to none made no appreciable difference. I wouldn't expect that to make any difference. > (Did I somehow set up Log4J2 incorrectly to get it to misbehave?) I don't think so. > Not sure what else I can do. Report it to the Log4J2 dev group perhaps? Ask on their users' list. It may be that when you use log4j2 in a webapp there is some clean-up method you need to call from a ServletContextListener. >> One thing worth noting. Yourkit offers two types of heap dumps. >> You don't want YourKit's own. There is a JVM bug that prevents >> some GC roots from being shown in that format and if your leak >> traces back to one of those you can end up scratching your head >> for days. >> >> Mark > > > Is there a likely chance that the bug interfered with my usage of Java Visual > VM and Eclipse Memory Analyzer and thus prevented me from determining the > real problem source? No. The bug just hides GC roots. If the only GC roots you see are for weak references then you might have hit this bug. Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
