Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-05-11 Thread qalab17
I'm also attempting to track down a cpu usage problem in tomcat. Can someone tell me how to generate thread dumps that includes the "cpu=" information like the ones Jesse attached? "ajp-8009-18" Id=187 in RUNNABLE cpu=242046 ms usr=242015 ms blocked 315 for -1 ms waited 256 for -1 ms When I crea

RE: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-02-02 Thread Caldarale, Charles R
> From: Christopher Schultz [mailto:ch...@christopherschultz.net] > Subject: Re: Tomcat 100% CPU usage after moving from Java 5 to 6 > > It's interesting that there's no timing data for that last statement. > It /is/ possible this is where the JVM went down. (Did

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-02-02 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jesse, On 2/2/2010 3:23 AM, Jesse Klaasse wrote: > Christopher Schultz-2 wrote: >> >> Can you give us a timestamp for when the trouble started? Reading GC >> output is pretty tedious. >> > > I believe the trouble started around 15 - 20 minutes before

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-02-02 Thread Dan Armbrust
> > Well, as I said, Tomcat is working much more stable now than it did before. > However, once in a few days it still becomes unresponsive, while CPU usage > stays incredibly low. Unresponsive, meaning what, exactly? If the CPU is idle, but your webapp isn't returning a response... it seems to i

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-02-02 Thread Jesse Klaasse
Christopher Schultz-2 wrote: > > Can you give us a timestamp for when the trouble started? Reading GC > output is pretty tedious. > I believe the trouble started around 15 - 20 minutes before the end of the gc log, so that would be from ~61h36m. Christopher Schultz-2 wrote: > > Also, what h

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-02-01 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jesse, On 2/1/2010 5:22 AM, Jesse Klaasse wrote: > > In the meantime, the environment has been much more stable than it was before > the move to Java 6 and the new JVM options. However, a few minutes ago, > Tomcat became completely unresponsive, whil

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-02-01 Thread Jesse Klaasse
In the meantime, the environment has been much more stable than it was before the move to Java 6 and the new JVM options. However, a few minutes ago, Tomcat became completely unresponsive, while the CPU usage was very low (~3%). This was on our external server, with these options: -Djavax.sql.Dat

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-26 Thread Peter Crowther
2010/1/26 Leon Rosenberg > Another customer of mine was playing with gc settings for nearly a > year, because his tomcats collection times went higher and higher, and > after a short look at the heap dump with jmap and jhat, we found out > that he had 100.000.000 uncollectable garbage objects in

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-26 Thread Leon Rosenberg
On Tue, Jan 26, 2010 at 4:50 PM, Jesse Klaasse wrote: > > Hi Chuck, thanks for your extensive comments about the JVM options.. > On a site note... I have made the observation that the JVM is performing best without any explicit gc settings. Most time explicit GC settings just cause havoc (fixes s

RE: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-26 Thread Jesse Klaasse
Hi Chuck, thanks for your extensive comments about the JVM options.. n828cl wrote: > >>-Xms10240m -Xmx10240m > > That's a very large heap; shouldn't hurt as long as you have enough RAM to > support it all without paging. (But see below for use with compressed > OOPs.) > This shouldn't be a

RE: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-26 Thread Caldarale, Charles R
> From: Jesse Klaasse [mailto:jesse.klaa...@indicia.nl] > Subject: Re: Tomcat 100% CPU usage after moving from Java 5 to 6 > > I found the following article particularly useful: > http://confluence.atlassian.com/display/DOC/Garbage+Collector+Performance+Issues Unfortunately, tha

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-26 Thread Jesse Klaasse
I also added the -XX:+UseCompressedOops JVM option (as suggested in the Confluence article I mentioned earlier). BTW, for those who want to know: this option is available since update 14 of JDK 6 (see also http://java.sun.com/javase/6/webnotes/6u14.html), you don't need the "Performance branch" as

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-26 Thread André Warnier
Jesse Klaasse wrote: I have now fixed the infinite loop, and again did the update from Java 5 to Java 6. This time all seems to be working like a charm! Along with fixing the infinite loop, I have dug a little deeper into the JVM arguments. I found the following article particularly useful: http

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-26 Thread Jesse Klaasse
I have now fixed the infinite loop, and again did the update from Java 5 to Java 6. This time all seems to be working like a charm! Along with fixing the infinite loop, I have dug a little deeper into the JVM arguments. I found the following article particularly useful: http://confluence.atlassia

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jesse, On 1/25/2010 8:25 AM, Jesse Klaasse wrote: > I don't know what you mean exactly by "re-intrant" Re-entrant (never seen it as "re-intrant") means that more than one thread of execution might be running the same code concurrently. I've never hea

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread David kerber
Jesse Klaasse wrote: Hi Leon, I don't know what you mean exactly by "re-intrant", but your comment points out I have created an endless while-loop! Apart from moving to Java 6, I had commented out some lines to reduce logging, not noticing I had created an endless loop by doing so. I only did th

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread André Warnier
Jesse Klaasse wrote: ... ctory -Dcom.sun.management.jmxremote -XX:MaxPermSize=512m -Xloggc:D:\logs\gc\tomcat-gc.log -XX:+PrintGCDetails -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -Xms4096m -Xmx10240m Apart from your basic

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Jesse Klaasse
Hi Leon, I don't know what you mean exactly by "re-intrant", but your comment points out I have created an endless while-loop! Apart from moving to Java 6, I had commented out some lines to reduce logging, not noticing I had created an endless loop by doing so. I only did this in a jar compiled f

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Leon Rosenberg
Hello Jesse, is the Workflow re-intrant? If yes: this is odd: you are iterating in a re-intrant action (i assume the action is re-intrant) over a shared variable? Vector vecOids = new Vector(); in the method: Iterator oidIterator = vecOids.iterator();

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Jesse Klaasse
Jesse Klaasse wrote: > > Those Workflow and DQWorkflow classes are from our company, written by the > programmer who worked on this project before me. I have attached the two > files for further examination. > I have noticed the performWorkflowAction method in DQWorkflow is not > synchronized an

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Jesse Klaasse
Leon Rosenberg-3 wrote: > > they are all seem to be either in an infinite loop: > at java.util.AbstractList$Itr.hasNext(AbstractList.java:339) > at > nl.indicia.vip.framework.util.Workflow.performWorkflowAction(Workflow.java:166) > at > nl.indicia.vip.framework.util.DQWorkflow.

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Jesse Klaasse
Pid Ster wrote: > > Have you tested those memory and garbage collection settings with this > version of Java? > Well, actually I haven't touched the settings which I used using Java 5. But, I have tested the settings to some extent. My development environment worked perfectly fine using the s

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Leon Rosenberg
P.S. Your tomcat is writing gc logs, please attach them next time. On Mon, Jan 25, 2010 at 12:31 PM, Leon Rosenberg wrote: > take a look on threads > ajp-8009-2 > ajp-8009-18 > ajp-8009-30 > ajp-8009-38 > > they are all seem to be either in an infinite loop: >        at java.util.AbstractList$Itr

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Leon Rosenberg
take a look on threads ajp-8009-2 ajp-8009-18 ajp-8009-30 ajp-8009-38 they are all seem to be either in an infinite loop: at java.util.AbstractList$Itr.hasNext(AbstractList.java:339) at nl.indicia.vip.framework.util.Workflow.performWorkflowAction(Workflow.java:166) at nl.

Re: Tomcat 100% CPU usage after moving from Java 5 to 6

2010-01-25 Thread Pid
On 25/01/2010 11:08, Jesse Klaasse wrote: I am running a production environment for a website (over half a million hits per day), using Tomcat 5.5.20 (I'm stuck to that version due to support restrictions) behind IIS 6 using JK connector 1.2.28, tcnative-1.dll (1.1.19) on Windows Server 2003 R2 E