On Wed, Jan 28, 2009 at 10:42 AM, Pieter Temmerman <ptemmerman....@sadiel.es> wrote: > > >>Have you found any java.lang.Thread.State: RUNNABLE threads? They are >>usually more interesting if it comes to a high cpu :-) > > These are the RUNNABLE threads on Tomcat 6: > > "RMI TCP Connection(42)-173.x.x.x" - Thread t...@112 > java.lang.Thread.State: RUNNABLE
reading from socket, usually not a problem. > "http-8081-35" - Thread t...@71 > java.lang.Thread.State: RUNNABLE > at java.lang.String.equals(String.java:1018) > at > com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:554) > at > org.codehaus.xfire.soap.handler.ReadHeadersHandler.invoke(ReadHeadersHandler.java:44) > at > org.codehaus.xfire.handler.HandlerPipeline.invoke(HandlerPipeline.java:131) > at > org.codehaus.xfire.transport.DefaultEndpoint.onReceive(DefaultEndpoint.java:64) > at > org.codehaus.xfire.transport.AbstractChannel.receive(AbstractChannel.java:38) > at > org.codehaus.xfire.transport.http.XFireServletController.invoke(XFireServletController.java:304) > at > org.codehaus.xfire.transport.http.XFireServletController.doService(XFireServletController.java:129) > at > org.codehaus.xfire.transport.http.XFireServlet.doPost(XFireServlet.java:116) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:710) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) > at This one actually looks like a problem, was the server under load as you created the dump, or did you removed the load first? If there were no external requests to the server, you most probably have an infinite loop somewhere (how stable is xfire anyway?) or you have a really really really large xml file to parse. If there were requests to the server as you took the dump, you might just happen to catch this part of the execution. In this case you should remove the traffic first and create a dump afterwards (of course only if the load remains after the traffic is cut off) > "http-8081-31" - Thread t...@67 > java.lang.Thread.State: RUNNABLE > at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl > $PrologDriver.next(XMLDocumentScannerImpl.java:930) > at > com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648) > at > com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140) > at > com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:548) > at > org.codehaus.xfire.soap.handler.ReadHeadersHandler.invoke(ReadHeadersHandler.java:44) > at > org.codehaus.xfire.handler.HandlerPipeline.invoke(HandlerPipeline.java:131) > at > org.codehaus.xfire.transport.DefaultEndpoint.onReceive(DefaultEndpoint.java:64) > at > org.codehaus.xfire.transport.AbstractChannel.receive(AbstractChannel.java:38) > at > org.codehaus.xfire.transport.http.XFireServletController.invoke(XFireServletController.java:304) > at > org.codehaus.xfire.transport.http.XFireServletController.doService(XFireServletController.java:129) > at > org.codehaus.xfire.transport.http.XFireServlet.doPost(XFireServlet.java:116) > note that the XFire path is almost identical. Personally I neved managed to catch healthy code in something like String.equals(), but it may be possible, even imho unlikeable. > > The Tomcat 5.5 webapp which connects to the webservice is not > responding. I saw the following thread on Tomcat 5.5 which may be the > connection to the webservice: > > The fact that it's reading from a socket, which I think it the > connection to the webservice, may explain why the application is not > doing anything. Yep > >>Also, as David posted, what is the HEAP usage? it's usually at the end >>of the dump. > > vdebian2:/usr/local/jdk1.6.0_07/bin# ./jmap 8100 > Attaching to process ID 8100, please wait... > Debugger attached successfully. > Server compiler detected. > JVM version is 10.0-b23 > > using thread-local object allocation. > Parallel GC with 2 thread(s) > > Heap Configuration: > MinHeapFreeRatio = 40 > MaxHeapFreeRatio = 70 > MaxHeapSize = 1073741824 (1024.0MB) > NewSize = 1048576 (1.0MB) > MaxNewSize = 4294901760 (4095.9375MB) > OldSize = 4194304 (4.0MB) > NewRatio = 8 > SurvivorRatio = 8 > PermSize = 16777216 (16.0MB) > MaxPermSize = 268435456 (256.0MB) > > Heap Usage: > PS Young Generation > Eden Space: > capacity = 58851328 (56.125MB) > used = 18383664 (17.532028198242188MB) > free = 40467664 (38.59297180175781MB) > 31.237466722925948% used > >From Space: > capacity = 327680 (0.3125MB) > used = 131072 (0.125MB) > free = 196608 (0.1875MB) > 40.0% used > To Space: > capacity = 393216 (0.375MB) > used = 0 (0.0MB) > free = 393216 (0.375MB) > 0.0% used > PS Old Generation > capacity = 477233152 (455.125MB) > used = 86499376 (82.49223327636719MB) > free = 390733776 (372.6327667236328MB) > 18.125181714115286% used > PS Perm Generation > capacity = 81395712 (77.625MB) > used = 62345160 (59.45697784423828MB) > free = 19050552 (18.16802215576172MB) > 76.59514054008152% used The dump looks healthy, but is it the dump from the tomcat 5.5 or tomcat 6? Because if tomcat 6 is the problem and the dump is from tomcat 5 its useless :-) > > I'm quite sure that one of the RUNNING threads of Tomcat 6 is the > problem, which will probably be related to the webservice. > But I'm puzzled with wath the problem might be. you need more research :-) For now it looks like your first tomcat is waiting for the second tomcat which just can't handle the requests as fast as its expected. You may want to check configuration options of both (threads in the connector etc). Also the heap dump of the second tomcat would be useful (if this one is from first). And a thread dump shortly after the traffic has gone (for example kill first tomcat, wait 1 minute, make thread dump). regards Leon --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
