Hi Ken (and Mark), Different "Mark" here... I'm new to this mailing list and am not a Tomcat developer. Forgive me if I'm interrupting your thread, but I am very interested in this topic, since I've spent a fair amount of time debugging OOM-PermGen errors within Tomcat (5.5.x). I would be interested in hearing any tips or tricks regarding OM-PermGen, too, and am happy to share my experiences regarding this.
None of the issues I've looked into have never been attributed to Tomcat. They have all been application issues related to an inability to garbage collect a webapp's classloader. This root causes were split between one of the following: * A webapp registering an object with another object that outlives the webapp and forgetting to unregister it webapp shutdown. As a result, that object cannot be garbage collected, which also prevents that webapp's classloader from being garbage collected. Examples of things that outlive a webapp or objects from libraries within Tomcat's shared library folder, Tomcat itself, and the JRE itself. For example, if a webapp registers a JDBC driver or a JCA/JCE provider, it should take care to unregister it on shutdown. * Instantiating a new Thread somehow (e.g. directly via its constructor, or indirectly via a thread pool, a Timer, or a ScheduledExecutorService) in response to a request to a webapp. This is because, by default, the new Thread inherits the parent thread's context classloader, which in Tomcat (5.5.x) is set to that webapp's classloader. If that Thread keeps the context classloader reference and outlives the webapp, the webapp's classloader cannot be garbage collected. As an aside, those causes highlight a reason why people should be weary of putting a new JAR files into one of Tomcat's shared library folders. Doing so requires that a webapp be vigilant about cleaning up any interactions with that library on shutdown. Said library also needs to provide a means to do the cleanup, as well as be sensitive to context classloader issues. When trying to debug these issues, I typically load things up in a profiler, stop the webapp and see of its classloader still resides in memory. When it exists, I use the profiler's "find the garbage collection root" to help determine why the classloader still resides in memory and figure out how to fix the issue. If using JProfiler (and probably other profilers), I think you have to tell it to go not stop when it reaches the Class object along the reference path. To help pro-actively detect webapp classloader garbage collection issues, I've leveraged the Insane library (a library I found from Netbeans while researching the topic) to write a utility that searches for webapp classloaders that should have been garbage collected. Using the utility in combination with automated tests has been definitely helped catch and diagnose issues. Apologies if this wasn't helpful. Please let me know if I'm wrong, outdated, or if there is a better way! Best regards, Mark DeSpain -----Original Message----- From: Ken Bowen [mailto:kbo...@als.com] Sent: Tuesday, April 21, 2009 1:33 PM To: Tomcat Users List Subject: Headstart on "Resolving OOM-PermGen errors on webapp reload" Mark, Any chance we could make a headstart on "Resolving OOM-PermGen errors on webapp reload" ?? Perhaps some general pointers, guidance etc. [to help you refine the talk in advance :-) ] The manager app is giving me more & more of: FAIL - Application at context path /ctx could not be started FAIL - Encountered exception java.lang.OutOfMemoryError: PermGen space This is on a Tomcat 6.0.18 (java 1.5) running on a remote Linux CentOS 5, in a Parallels VM, with httpd + mod_jk, running two versions of the app. Total memory available is 432 MB, with a number of non-system, non-TC processes, including ActiveMQ and 9 Postres processes. But there appears to be about 200MB of unused memory according to the Parallels control panel. I develop locally on a Mac OS/X 10.5.6 box with 4GBmem using (My)Eclipse and Tomcat 6.0.18 directly. With lots & lots of reloads, I'm not surprised that I eventually hit OOM PermGen space in this setting, but it happens much less often than on the CentOS box (of course 4GB != 432MB). Thanks...Ken Mark Thomas wrote: > I was at a conference recently and rather rashly made the statement > "OOM > PermGen errors on reload are an application issue, not a Tomcat one". > Several people came up to me afterwards with variations on a theme of > "Our app has OOM on reload - put your money where your mouth is and > show > us where we have gone wrong" > > Having helped several people track down the causes of these errors > (and > yes they were all application issues) it occurred to me that there may > be interest in this at ApacheCon. So my suggestion is: > > Title: Resolving OOM-PermGen errors on webapp reload > Type: Session > Abstract: What causes the error. Typical root causes. > How to find and fix the root causes. With live demo. > Audience: Developer/SysAdmin > > Thoughts? > > Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
