-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris,
On 3/4/16 2:13 PM, Chris Brown wrote: > Tomcat version: 7.0.42 Server OS: Sun Solaris UCP Version: Oracle > 12c Oracle Driver: ojdbc7 That doesn't look like an Oracle JDBC driver version I've seen before. Can you clarify? > JDK version: 1.8.0_31 You might want to upgrade Tomcat and Java. Those are somewhat old. > Issue Description: When tomcat shuts down our web service during a > hot deployment threads are left running that tomcat cannot > shutdown. I believe the threads are from the universal connection > pool. We use oracle's universal connection pool (ucp.jar). After > multiple deployments the threads build up and cause high CPU usage > from the 2 tomcat threads. > > Log Error Examples: Feb 08, 2016 8:00:25 PM > org.apache.catalina.loader.WebappClassLoader > clearReferencesThreads SEVERE: The web application [/RatingService] > appears to have started a thread named [UCP-worker-thread-3] but > has failed to stop it. This is very likely to create a memory > leak. Feb 08, 2016 8:00:25 PM > org.apache.catalina.loader.WebappClassLoader > clearReferencesThreads SEVERE: The web application [/RatingService] > appears to have started a thread named [Thread-438] but has failed > to stop it. This is very likely to create a memory leak. Feb 27, > 2016 6:15:14 PM org.apache.catalina.loader.WebappClassLoader > clearReferencesThreads SEVERE: The web application [/RatingService] > appears to have started a thread named > [oracle.jdbc.driver.BlockSource.ThreadedCachingBlockSource.BlockReleas er] > but has failed to stop it. This is very likely to create a memory > leak. Feb 27, 2016 6:15:14 PM > org.apache.catalina.loader.WebappClassLoader > clearReferencesThreads SEVERE: The web application [/RatingService] > appears to have started a thread named > [ForkJoinPool.commonPool-worker-32] but has failed to stop it. This > is very likely to create a memory leak. Feb 27, 2016 6:15:14 PM > org.apache.catalina.loader.WebappClassLoader > clearReferencesThreads SEVERE: The web application [/RatingService] > appears to have started a thread named [Timer-121] but has failed > to stop it. This is very likely to create a memory leak. > > Correction made to code to try and fix issue: I added a > ServletContextListener to our web service. On the contextDestroyed > method we destroy the connection pools. Our web service uses 2 > connection pools. How exactly are you destroying the connection pools? > I also upgraded to Tomcat 8 and the issue still occurred, but the > messages are now warnings instead of SEVERE errors. That's just an effect of Tomcat downgrading the log level under which it logs those issues. > I've searched the web and have tried many suggestions that have > not helped except destroying the connection pools, but the issue > still occurs. Please help!!! After you get those messages in the log, if you do a thread dump on the JVM, do you still see those threads running? Generally speaking, Tomcat should be able to shut down even with the above errors. Hopefully, none of those threads are non-daemon threads, meaning that they shouldn't stop the JVM from stopping once the JVM's main thread terminates. If your web application leaves those threads running after deployment, then multiple re-deployments without a Tomcat/JVM restart will likely result in a large memory leak. (Those threads will pin the web application's ClassLoader in memory along with all the classes it ever loaded, and these will pile-up until you have exhausted your permgen or other heap spaces). > [UCP-worker-thread-3] This is almost certainly something form Oracle's UCP > [Thread-438] This could be anything. A thread dump could tell you what it might be. > [oracle.jdbc.driver.BlockSource.ThreadedCachingBlockSource.BlockReleaser ] Obviously, this is a thread from Oracle's JDBC driver. > [ForkJoinPool.commonPool-worker-32] Maybe something from UCP? Not sure. > [Timer-121] This is usually a thread started by TimerTask, a stock Java class. But any component may have created this thread. It might be worth searching your own code to see if a TimerTask is being launched and never shut-down when your web application is shutting down. Make arrangements to make sure that you shut these down when the application is going out of service. For the Oracle-specific threads, read the documentation or search online for how to stop those threads as the application is coming down. There could be a bug in the library(ies) that might be fixed in a more recent version. - -chris -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlbZ/woACgkQ9CaO5/Lv0PBwvwCeLblpAKfIen1G0nmZvoqkLgWe qCoAnR84mW3an4+BOVKvzuQTI2fTTodx =5+gA -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
