Chuck and Dan,
First, some housekeeping to respond to your points and questions:
1. The 3.6GB I stated in my prior email to Dan was the total memory used.
The Tomcat process (from top) was 1.7-1.8GB.
2. I did not know that Linux cached disk accesses in RAM. I have worked
with Linux (and Unix and Aix before that) for a number of years but never
understood all the pieces... as long as it worked, I was happy (and, in
general, I didn't ask much of the systems.)
To sum up your observations: The memory usage I was seeing was not unusual
and not likely the source of the problem.
The system failed again about 30 minutes ago. The overall memory dropped to
2.9GB (from top.)
The last entries in catalina.out were:
41608.326: [GC 41608.326: [ParNew: 72021K->4186K(76672K), 0.0674360 secs]
376022K->309373K(1040064K) icms_dc=0 , 0.0675430 secs] [Times: user=0.25
sys=0.00, real=0.07 secs]
41610.048: [GC 41610.048: [ParNew: 72346K->8512K(76672K), 0.0408460 secs]
377533K->318233K(1040064K) icms_dc=0 , 0.0409480 secs] [Times: user=0.15
sys=0.00, real=0.04 secs]
The GC information in catalina.out before that was:
41391.749: [GC 41391.749: [ParNew: 70179K->2466K(76672K), 0.0064220 secs]
374180K->306467K(1040064K) icms_dc=0 , 0.0065240 secs] [Times: user=0.02
sys=0.00,
real=0.01 secs]
41393.383: [GC 41393.383: [ParNew: 70626K->2571K(76672K), 0.0059940 secs]
374627K->306572K(1040064K) icms_dc=0 , 0.0060940 secs] [Times: user=0.03
sys=0.00,
real=0.01 secs]
and before that:
41216.669: [GC 41216.669: [ParNew: 69204K->2621K(76672K), 0.0068300 secs]
367414K->300832K(1040064K) icms_dc=0 , 0.0069380 secs] [Times: user=0.02
sys=0.00,
real=0.00 secs]
41216.710: [GC 41216.710: [ParNew: 70720K->3097K(76672K), 0.0064720 secs]
368930K->301364K(1040064K) icms_dc=0 , 0.0065740 secs] [Times: user=0.03
sys=0.01,
real=0.00 secs]
41216.750: [GC 41216.750: [ParNew: 71117K->3241K(76672K), 0.0063760 secs]
369384K->301618K(1040064K) icms_dc=0 , 0.0064780 secs] [Times: user=0.03
sys=0.00,
real=0.00 secs]
The /var/log/messages from that machine for today were:
Feb 4 00:08:05 tomcat2 -- MARK --
Feb 4 00:28:05 tomcat2 -- MARK --
Feb 4 00:48:05 tomcat2 -- MARK --
Feb 4 01:08:05 tomcat2 -- MARK --
Feb 4 01:28:05 tomcat2 -- MARK --
Feb 4 01:48:05 tomcat2 -- MARK --
Feb 4 02:08:05 tomcat2 -- MARK --
Feb 4 02:28:05 tomcat2 -- MARK --
Feb 4 02:48:05 tomcat2 -- MARK --
Feb 4 03:08:05 tomcat2 -- MARK --
Feb 4 03:28:05 tomcat2 -- MARK --
Feb 4 03:48:05 tomcat2 -- MARK --
Feb 4 04:08:05 tomcat2 -- MARK --
Feb 4 04:28:06 tomcat2 -- MARK --
Feb 4 04:48:06 tomcat2 -- MARK --
Feb 4 05:08:06 tomcat2 -- MARK --
Feb 4 05:28:06 tomcat2 -- MARK --
Feb 4 05:48:06 tomcat2 -- MARK --
Feb 4 06:08:06 tomcat2 -- MARK --
Feb 4 06:28:06 tomcat2 -- MARK --
Feb 4 06:48:06 tomcat2 -- MARK --
Feb 4 07:08:06 tomcat2 -- MARK --
Feb 4 07:28:06 tomcat2 -- MARK --
Feb 4 07:48:06 tomcat2 -- MARK --
Feb 4 08:08:06 tomcat2 -- MARK --
Feb 4 08:28:06 tomcat2 -- MARK --
Feb 4 08:48:06 tomcat2 -- MARK --
Feb 4 09:08:06 tomcat2 -- MARK --
Feb 4 09:28:06 tomcat2 -- MARK --
Feb 4 09:33:32 tomcat2 /usr/sbin/gpm[3414]: *** info [mice.c(1766)]:
Feb 4 09:33:32 tomcat2 /usr/sbin/gpm[3414]: imps2: Auto-detected
intellimouse PS/2
Feb 4 09:48:06 tomcat2 -- MARK --
Feb 4 10:08:06 tomcat2 -- MARK --
Feb 4 10:28:06 tomcat2 -- MARK --
Feb 4 10:48:06 tomcat2 -- MARK --
Feb 4 11:08:06 tomcat2 -- MARK --
Feb 4 11:28:06 tomcat2 -- MARK --
Feb 4 11:48:06 tomcat2 -- MARK --
Feb 4 12:08:06 tomcat2 -- MARK --
Feb 4 12:28:06 tomcat2 -- MARK --
Feb 4 12:48:06 tomcat2 -- MARK --
Feb 4 13:08:07 tomcat2 -- MARK --
Feb 4 13:28:07 tomcat2 -- MARK --
Feb 4 13:48:07 tomcat2 -- MARK --
I don't see anything here and this file has looked pretty much like this
every failure.
JAVA_OPTS in catalina.sh are:
JAVA_OPTS="-Xms1024m -Xmx1024m -XX:PermSize=384m -XX:MaxPermSize=384m -XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/usr/local/tomcat/logs"
I checked /usr/local/tomcat/logs but found only the usual files. The
information was in catalina.2010-02-04.log showed our nightly restart of
Tomcat but nothing more recent. There were, however, several entries like
the following that indicate some kind of problem:
Feb 4, 2010 1:10:03 AM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type
[java.lang.ThreadLocal] (value [java.lang.threadlo...@68758d7b]) and a value
of type [null]
(value [net.sf.jasperreports.engine.export.legacy.borderoffse...@7b47951c])
but failed to remove it when the web application was stopped. To prevent a
memory
leak, the ThreadLocal has been forcibly removed.
Should I be concerned about this problem right now?
Any ideas?
----- Original Message -----
From: "Caldarale, Charles R" <chuck.caldar...@unisys.com>
To: "Tomcat Users List" <users@tomcat.apache.org>
Sent: Thursday, February 04, 2010 12:46 PM
Subject: RE: Tomcat dies suddenly
From: Carl [mailto:c...@etrak-plus.com]
Subject: Re: Tomcat dies suddenly
if I understand it correctly, Tomcat is (mostly) running in
the heap and should, therefore, not be requiring more memory
Not necessarily. The real memory used by a Java process can continue to
climb until all the pages assigned to the heap and associated spaces have
actually been touched. That might not happen for quite some time,
especially in PermGen.
However, that is slowly being eaten away by something.
I would have expected the usage to flatten out and then
fluctuate about that number. This implies a memory
leak someplace in the system.
Probably not. As Dan explained, that's normal file caching behavior.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you received
this in error, please contact the sender and delete the e-mail and its
attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
