Edited for sanity.
Comments surrounded by ==== COMMENT ====
On 8/27/2012 1:24 PM, Christopher Schultz wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Mark,
On 8/27/12 3:39 PM, Mark Eggers wrote:
This is long and mostly off-topic. Read if you wish to follow the
fun and games of tracking down permgen leaks, especially with a
third party obfuscated library.
:)
JMLPL documentation?
http://tomcat.apache.org/tomcat-6.0-doc/config/listeners.html#JRE_Memory_Leak_Prevention_Listener_-_org.apache.catalina.core.JreMemoryLeakPreventionListener
==== COMMENT ====
Roger that - JMLPL - java memory leak prevention listener - I should
have known.
==== COMMENT ====
I'm also stuck with 6.0.x (where x is 35 in development, and 20 in
production). I do not think that upgrading production is a
possibility.
However, I'm seeing this in 6.0.35.
I was referring to this, which was added in 6.0.34:
https://issues.apache.org/bugzilla/show_bug.cgi?id=51688
==== COMMENT ====
Ah, then it's a good thing I do this in the application, since the
target platform is 6.0.20.
==== COMMENT ====
It's the same thing you do in your own listener (unless
getSystemEventQueue() does something even worse than simply calling
getDefaultToolkit()).
==== COMMENT ====
All this really does is start the AWT Event Thread. This seems to work
fine for both Windows and various flavors of Linux (Fedora, CentOS). It
appears to work both with a windowing environment, and without a
windowing environment running java.awt.headless=true. At least the
thread shows up in thread dumps.
==== COMMENT ====
This was done since the target server was not known (at the time).
Also, this doesn't require changes to a Tomcat server that is not
in my control.
Using the following leak prevention listener configuration didn't
change the fact that there was a leak on unloading the
application.
<Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
AWTThreadProtection="true"/>
Hmm... does that happen in 6.0.35 as well, or only 6.0.20? It was only
added in 6.0.34 but it should be working. If not, please re-open the
aforementioned bug and explain.
==== COMMENT ====
This does happen in both 6.0.35 and 7.0.27. I know, I know, I'll upgrade
to 7.0.29 soon.
==== COMMENT ====
Note that by default I have this configured in my environment.
<Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener"/>
According to the documentation, this takes care of many leak
issues, including AppContext.
Right.
==== COMMENT ====
Stack traces deleted to keep everyone sane
==== COMMENT ====
Hmm... that's ... not supposed to happen. The AppContext
leak-prevention has been in Tomcat since back in 2009. Can you confirm
that the JreMemoryLeakProtectionListener is actually being loaded?
==== COMMENT ====
A heap dump (jvisualvm) on the following environments show one instance
of the class being loaded.
1. Windows 7, JRE 1.7.0_05, Tomcat 6.0.35
2. Windows 7, JRE 1.7.0_05, Tomcat 7.0.27
3. Windows 7, JRE 1.6.0_32, Tomcat 6.0.35
4. Windows 7, JRE 1.6.0_32, Tomcat 7.0.27
5. CentOS 6.3, JRE 1.6.0_32, Tomcat 6.0.35, -Djava.awt.headless=true
this is actually running X Windows though
6. CentOS 5.8, JRE 1.6.0_32, Tomcat 6.0.35, -Djava.awt.headless=true
not running X Windows (dump via jmap)
Hmm - looking at Tomcat 6.0.35 on the CentOS 5.8 system, I see the
following when looking at the instance of JreMemoryProtectionListener.
appContextProtection (Z) : true
awtThreadProtection (Z) : true
classesToInitialize (L) : <null>
driverManagerProtection (Z) : true
gcDaemonProtection (Z) : true
ldapPoolProtection (Z) : true
securityLoginConfigurationProtection (Z) : true
securityPolicyProtection (Z) : true
tokenPollerProtection (Z) : true
urlCacheProtection (Z) : true
xmlParsingProtection (Z) : true
And I do have the attribute set to manage the AWT thread (at least on
the Tomcat I dumped).
On the Windows (Tomcat 6.0.35, JRE 1.6.0_32), I see the following:
appContextProtection (Z) : true
awtThreadProtection (Z) : false
classesToInitialize (L) : <null>
driverManagerProtection (Z) : true
gcDaemonProtection (Z) : true
ldapPoolProtection (Z) : true
securityLoginConfigurationProtection (Z) : true
securityPolicyProtection (Z) : true
tokenPollerProtection (Z) : true
urlCacheProtection (Z) : true
xmlParsingProtection (Z) : true
which is what I expect from server.xml.
Commenting out the listener (actually it doesn't matter either way since
I'm running in a windowed environment) and adding the AWT Event thread
protection produces on Windows:
appContextProtection (Z) : true
awtThreadProtection (Z) : true
classesToInitialize (L) : <null>
driverManagerProtection (Z) : true
gcDaemonProtection (Z) : true
ldapPoolProtection (Z) : true
securityLoginConfigurationProtection (Z) : true
securityPolicyProtection (Z) : true
tokenPollerProtection (Z) : true
urlCacheProtection (Z) : true
xmlParsingProtection (Z) : true
And I still see the exact same permgen memory leak upon unloading the
application.
The problem looks to be in an internal use of MediaTracker. I suspect
that the library uses this to speed image recovery should there be
repeated calls.
Unfortunately, these images are dynamically generated, so any use of
MediaTracker is pretty worthless in this use case.
There's a way to purge the library caches, and I've experimented with
that as well. Unfortunately, it doesn't unregister the images it stored
in the MediaTracker. In short, flushing the library's cache in this use
case does not address the problem.
. . . thanks for your feedback Chris.
/mde/
==== COMMENT ====
- -chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
