Mark Thanks again for taking the time to educate me and answer my questions. My lengthy replies below. To summarize, our app does not seem to commit any egregious memory leak offenses, from what I can tell so far. I plan to heap dump an older version of the app that does not use third-party libraries for comparison purposes.
-----Original Message----- From: Mark Thomas [mailto:ma...@apache.org] Sent: Tuesday, July 19, 2016 3:15 PM To: Tomcat Users List Subject: Re: OutOfMemoryError: PermGen space > On 19/07/2016 17:19, Berneburg, Cris J. - US wrote: > > <snip/> > > >> This is probably a useful read: > >> http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks > >> -60mins.pdf > >> > >> Despite the age, it is still very relevant today. > > > > > > Thanks for reminding me about that document. It does sound relevant. > > The document says, > > > >> How memory leaks occur: Reference chains [...] Retaining a reference > >> to a single object from a web application pins every class loaded by > >> the web application in the Permanent Generation > > > > Practically speaking, how is a reference retained? > > 1. Is a module-level (member) variable in a Servlet an example? > > No. Assuming the class that defines the Servlet is packaged with the > web application (i.e. in a JAR in WEB-INF/lib or in WEB-INF/classes All our servlets are classes in WEB-INF/classes. > > 2. How about using ServletContext.setAttribute without an accompanying > > removeAttribute? > > No. > > > 3. I see in one of my listeners some member (module-level) variables: > > a private static for the logger and also a protected final String. > > Is that a problem? > > No. Dang, I was hoping that would be it. :-) > > If you could provide a little more guidance about the details, I would > > appreciate it. > > Sure. > > There are two categories of objects we are concerned about. Objects > defined by classes that are provided by your web application (in > WEB-INF/lib or WEB-INF/classes) and objects defined by classes that > are provided by the container (Tomcat) or the JVM. > > References from web application objects to web application objects > are fine. > > References from web application objects to container objects are fine. > > References from container objects to web application objects are > potentially a problem. > > Additionally, references from container objects to the web application > class loader are potentially a problem. Hmm... I'm gonna need some time to absorb that. > For example: > > When you start a thread in a web application, the thread context > class loader will be the web application class loader and the thread > holds a reference to this. If the thread is not stopped when the web > application is stopped there will be a reference held to the web > application class loader which will cause a leak. To my knowledge we are not explicitly starting any new threads. Our app is pretty basic. > If you add a logging framework to Tomcat's lib directory and then > package a custom log formatter with your web application, the formatter > will be registered with the logging framework and unless you unregister > it when the application stops there will be reference from logging > framework to custom formatter object to custom formatter class to web > application class loader which will cause a leak. We did not program a custom formatter with LOG4J2, just used a format parm in the config. Also, LOG4J jar files were added the to the app WEB-INF/lib folder, not Tomcat's. Sounds like there are a lot of potential sources for memory leaks. I'm either going to have to keep digging deeper using a heap dump tool or simply punt without figuring it out. Will a heap dump analyzer reveal what instances are in PermGen space and what has references to them? Maybe I can follow up a chain of references to determine the source. In the heap dump the largest collection of instances is java.lang.String. Could concatenation by assignment be a problem? Example, abc = abc + xzy. While we are guilty of that, performing a heap dump on a fresh, idle, unused app would not have called the offending code. So that cannot be it. The second and third largest groups of instances in the heap dump are char[] and java.util.HashMap$Entry. They must be from a library or third-party class, since not one char or HashMap instance is declared in our entire application. As an experiment, I may grab heap dumps from and restart an older version of the app that does not use Mybatis or Log4J to see how that affects the top 3 class instances. If the sizes are significantly smaller, then perhaps the leaks are from those libraries. FYI, I was pretty careful in taking the previous advice from this group about using the try-open/catch/finally-close paradigm when opening and closing each DB connection from Mybatis to prevent connection-related memory leaks. My intent was that connection references are not held onto by the app and are only potentially retained by the Mybatis connection pool. Hmm... Mybatis does not seem to have an explicit call to shut down, something to be called in ServletContextListener.contextDestroyed, for instance. > HTH, > > Mark > > P.S. I'll be talking about this (hopefully - if my talk gets accepted) > at ApacheCon EU later this year. Well, I'm looking forward to seeing your talk in the archives. ;-) -- Cris Berneburg, Lead Software Engineer, CACI --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
