In process... thanks for the suggestion.
Carl
----- Original Message -----
From: "Peter Crowther" <peter.crowt...@melandra.com>
To: "Tomcat Users List" <users@tomcat.apache.org>
Sent: Wednesday, January 13, 2010 8:49 AM
Subject: Re: Tomcat dies suddenly (was JVM goes away)
Very difficult to know what the problem is. One thing you can now do
(as you've switched to another production server) is to run a memory
test across the "bad" server. A T110 doesn't use error-correcting
memory, as I recall, so a dodgy bit could cause problems. Give it a
couple of hours with memtest86+ and you'll at least know whether
you've been chasing phantoms due to a hardware error.
(I'm perhaps biased - I've had memory errors on three low-end servers now)
- Peter
2010/1/13 Carl <c...@etrak-plus.com>:
From the original posting:
This is a new server, a Dell T110 with a Xeon 3440 processor and 4GB
memory. I have turned off both the turbo mode and hyperthreading.
The environment:
64 bit Slackware Linux
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)
Tomcat: apache-tomcat-6.0.20
These are the current
JAVA_OPTS="-Xms1024m -Xmx1024m -XX:PermSize=368m -XX:MaxPermSize=368m"
In the previous posting, I noted that I have observed the memory usage and
general performance with Java VisualVM and have seen nothing strange. GC
seems to be performing well and the memory rarely gets anywhere near the
max. New information: I thought I was seeing GC as memory usage was going
up and down but in fact it was mostly people coming onto the system and
leaving it. After several hours, the memory settles to a baseline of about
375MB. Forced GC never takes it below that value and the ups and downs
from the people coming onto and leaving the system also returns it to
pretty much that value. The maximum memory used never was above 700MB for
the entire day.
The server runs well, idling along at 2-5% load, except for a quick spike
during GC, serving jsp's, etc. at a reasonable speed. Without warning and
with no tracks in any log (Tomcat or system) or to the console, the JVM
will just go away, disappear. New information: The JVM does not just go
away but somehow Tomcat shutsdown as the ports used by Tomcat are closed
(pointed out by Konstantin.) Sometimes, the system will run for a week,
sometimes for only several hours. Initially, I thought the problem was the
turbo or hyperthreading but, no, the problem persists.
When Tomcat shuts down, the memory that it held is still being held (as
seen from top) but it is nowhere near the machine physical memory.
The application has been running on an older server (Dell 600SC, 32 bit
Slackware, 2GB memory) for several years and, while the application will
throw exceptions now and then, it never crashed. This lead me to believe
the problem had something to do with the 64 bit JVM but, with without
seeing errors anywhere, I can't be certain and don't know what I can do
about it except go back to 32 bit.
New information.
Last evening, I observed the heap and permGen memory usage with Visual
JVM. It was running around 600MB before I forced a GC and 375MB afterward.
Speed was good. Memory usage from top was 2.4GB. Five minutes later,
Tomcat stopped leaving no tracks that I could find. The memory usage from
top was around 2.4GB. The memory usage from Visual JVM was still showing
400MB+ although the Tomcat process was gone. I restarted Tomcat (did not
reboot) so Tomcat had been shutdown gracefully enough to close the ports
(8080, 8443, 443.) Tomcat stayed up for less than an hour (under light
load) and stopped again. The memory used according to top was less than
3GB but I didn't get the exact number. I restarted it again (no server
reboot) and it ran for the rest of the night (light load) and top was
showing 3.3GB for memory this morning.
I brought up a new server last night and have switched to that server for
production (same Linux, JDK, server.xml, JAVA_OPTS, etc.). It would seem
if the problem is with my application or the JVM, that the problem will
follow me to the new server.
Anyone have any ideas how I might track this problem down?
Thanks,
Carl
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
