Thanks Filip, oversight on my part, here we go:
[EMAIL PROTECTED]:/logs/tomcat> ps -eL -o pid,%cpu,lwp | grep -i 4046 | grep
-iv 0.0
4046 0.6 4047
4046 0.1 4052
4046 0.1 4053
4046 21.9 4078
4046 18.7 4108
4046 0.1 4109
"http-8080-Poller-0" daemon prio=1 tid=0x0000002ae2f860e0 nid=0xfee runnable
[0x00000000412cf000..0x00000000412cfc10]
at java.util.HashMap.newKeyIterator(HashMap.java:889)
at java.util.HashMap$KeySet.iterator(HashMap.java:921)
at java.util.HashSet.iterator(HashSet.java:154)
at sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:127)
- locked <0x0000002ab2ac2810> (a java.util.HashSet)
at sun.nio.ch.PollSelectorImpl.doSelect(PollSelectorImpl.java:60)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x0000002ab2ac2870> (a sun.nio.ch.Util$1)
- locked <0x0000002ab2ac2858> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000002ab2ab5c80> (a sun.nio.ch.PollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1417)
at java.lang.Thread.run(Thread.java:595)
"http-8080-exec-18" daemon prio=1 tid=0x0000002ae3c5d8e0 nid=0x100c runnable
[0x00000000430ec000..0x00000000430edb10]
at
org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:44)
at
org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:794)
at
org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
at
org.apache.coyote.http11.filters.GzipOutputFilter$FakeOutputStream.write(GzipOutputFilter.java:164)
at java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:95)
at
org.apache.coyote.http11.filters.GzipOutputFilter.end(GzipOutputFilter.java:122)
at
org.apache.coyote.http11.InternalNioOutputBuffer.endRequest(InternalNioOutputBuffer.java:396)
at
org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1080)
at org.apache.coyote.Response.action(Response.java:183)
at org.apache.coyote.Response.finish(Response.java:305)
at
org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:276)
at
org.apache.catalina.connector.Response.finishResponse(Response.java:486)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:287)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:887)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:696)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2009)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
----- Original Message ----
From: Filip Hanik
since you are running on linux, you know that you can get the id of the
thread that is taking up all the CPU, just use a binary top that let
you
list individual threads.
as you can see, the thread dump you have, doesn't really show anything,
you're simply assuming that it's that call taking up CPU, but if your
CPU usage is very high, then very little code is actually moving
through.
there are a few bugs with references, but until you get the actual
thread causing the CPU usage, then you wont know for sure
a few examples are, but nothing concrete.
http://issues.apache.org/bugzilla/show_bug.cgi?id=42090
http://issues.apache.org/bugzilla/show_bug.cgi?id=42925
gather up the data, get the thread id that is causing CPU, match that
with your thread dump, and then you will know for sure
Filip
Peter wrote:
> Hi
>
> We are having a problem with Tomcat 6 using the NIO (running on linux
with Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_12-b04, mixed
mode) that it consumes all CPU after a few hours in production, prior to
that we ran Tomcat 6 with AJP and Apache 2.0 with mod_jk in front of it
for over a month without any problems. While it is consuming all CPU it
still serves requests but obviously much slower. During the last
"episode" I collected some thread dumps over the period of 10 - 15 minutes
and found 3 runnable threads that were present in all the dumps and were
doing the exact same thing:
>
> "http-8080-exec-41" daemon prio=1 tid=0x0000002ae320dad0 nid=0x12ac
runnable [0x0000000045c18000..0x0000000045c18c10]
> at
org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:44)
> at
org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:794)
> at
org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
> at
org.apache.coyote.http11.filters.GzipOutputFilter$FakeOutputStream.write(GzipOutputFilter.java:164)
> at
java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:95)
> --
> "http-8080-exec-41" daemon prio=1 tid=0x0000002ae320dad0 nid=0x12ac
runnable [0x0000000045c18000..0x0000000045c18c10]
> at
org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:44)
> at
org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:794)
> at
org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
> at
org.apache.coyote.http11.filters.GzipOutputFilter$FakeOutputStream.write(GzipOutputFilter.java:164)
> at
java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:95)
> --
> "http-8080-exec-41" daemon prio=1 tid=0x0000002ae320dad0 nid=0x12ac
runnable [0x0000000045c18000..0x0000000045c18c10]
> at
org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:44)
> at
org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:794)
> at
org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
> at
org.apache.coyote.http11.filters.GzipOutputFilter$FakeOutputStream.write(GzipOutputFilter.java:164)
> at
java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:95)
>
> "http-8080-exec-29" daemon prio=1 tid=0x0000002ae4152d10 nid=0x6e3a
runnable [0x0000000043af7000..0x0000000043af7b90]
> at
org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:44)
> at
org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:794)
> at
org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
> at
org.apache.coyote.http11.filters.GzipOutputFilter$FakeOutputStream.write(GzipOutputFilter.java:164)
> at
java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:95)
>
> "http-8080-exec-16" daemon prio=1 tid=0x0000002ae39bf030 nid=0x18d1
runnable [0x0000000043cf9000..0x0000000043cf9e10]
> at
org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:44)
> at
org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:794)
> at
org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
> at
org.apache.coyote.http11.filters.GzipOutputFilter$FakeOutputStream.write(GzipOutputFilter.java:164)
> at
java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:95)
>
> etc. Here is the connector configurations:
>
> <Connector port="8080"
> connectionTimeout="20000"
> maxThreads="300"
> enableLookups="false"
> compression="5000"
> protocol="org.apache.coyote.http11.Http11NioProtocol"
> redirectPort="8443"
> keepAliveTimeout="5000"
> maxKeepAliveRequests="1000" />
>
> <Connector port="8443"
> protocol="org.apache.coyote.http11.Http11AprProtocol"
> maxThreads="300"
> minSpareThreads="25"
> maxSpareThreads="75"
> enableLookups="false"
> disableUploadTimeout="true"
> acceptCount="100"
> scheme="https"
> secure="true"
> SSLEnabled="true"
> SSLCertificateFile="xyz.com.crt"
> SSLCertificateKeyFile="xyz.com.key"
> SSLPassword="" />
>
>
> The server goes into this state very regularly, this configuration
has been in service for 3 days and it's goes into this state every 2 - 5
hours until restarted.
>
> Has anyone experience any similar behaviour? Any ideas or
suggestions?
>
> Thanks
> Peter
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: [email protected]
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: [email protected]
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To start a new topic, e-mail: [email protected]
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
