its very weird indeed. We had acted proactively on a tomcat which
started to show same symptoms (cpu load going higher (35%) but users
going down (means it replies slower than the other machines and the lb
gives less traffic to it) and after checking the logs, sent the QUIT
signal which worked this time (hence i can't be sure, that its
completely the same problem).
But the thread dump was weird, from
51 RUNNABLE threads
38 in at java.net.SocketInputStream.socketRead0(Native Method)
2 in write to file in logging
...
and two in
at java.util.ArrayList.size(ArrayList.java:177)
Now this is something i personally consider weird, since it's hard
enough to find a thread in any places in working (not waiting) code in
a thread dump, but twice in the same code line?
The calling code line (our code) is
public boolean isRoot() {
return pathElements.size() == 0;
}
We assume that the other tomcats didn't produce any thread dumps
because our trigger happy system admins just sent the kill -9 too soon
after the kill -3, and the jvm was overloaded. Well it's a wild guess,
but i have no better one yet (unless jvm really locked up internally,
but why??? we haven't change the jvm or tomcat or os or anything for a
year now).
But back to the ArrayList, how high is the probability of having
ArrayList.size() twice in the similar call-tree in one ThreadDump?
regards
Leon
On Mon, Dec 1, 2008 at 5:41 PM, Peter Crowther
<[EMAIL PROTECTED]> wrote:
>> From: Caldarale, Charles R [mailto:[EMAIL PROTECTED]
>> That's ugly. Sounds like either the OS failed to deliver the
>> signal, or the JVM is locked up internally (probably the
>> latter).
>
> That's rather what I was reckoning. I've had signal delivery fail before if
> all the threads were stuck in kernel code via system calls. This was SunOS
> 3.x, which shows my age - NFS was kernel-mode at that point, which made for
> interesting times if file accesses got stuck. Processes not responding to a
> kill -9 was a new one on me!
>
> Might the code have ended up with all threads stuck in OS calls? It feels a
> little unlikely, as I assume the JVM keeps a couple of threads for itself...
>
> - Peter
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
