gb, > Our tomcat was non-responsive, and catalina.out had several > 'java.lang.OutOfMemory' errors. So this isn't totally unknown to us > (our apps leak a bit of memory, plus we're running on a small JVM).
Don't forget that pretty much all bets are off of you get an
OutOfMemoryError. The JVM has to be considered unstable, so anything you
see might not indicate anything conclusive. It is unfortunately but
true: if you get an OutOfMemoryError, you must restart your application.
> I tried to shutdown tomcat and it wouldn't die.
This is not surprising. kill -9 is your only good option, I think.
> in Object.wait() [b2bfe000..b2bff8d8]
> at java.lang.Object.wait(Native Method)
> - waiting on <0x45251bb0> (a sun.misc.SoftCache)
> at java.lang.Object.wait(Object.java:429)
> at java.util.ResourceBundle.findBundle(ResourceBundle.java:862)
[snip]
> at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:442)
> at
> org.apache.coyote.tomcat5.CoyoteRequest.<init>(CoyoteRequest.java:151)
[snip]
> So, Tomcat code is waiting for some other code (my code) to release a lock?
Your code is nowhere to be found: this is all Tomcat and JDK stuff.
CoyoteRequest's constructor is trying to create a SimpleDateFormat
object, which is causing a lot of initialization code to run, which has
a lock on an object. I don't think there's anything you can do about this.
> - waiting on <0x45251bb0> (a sun.misc.SoftCache)
>
> Is that the correct interpretation?
I think it's unlikely that your code is the problem, though it certainly
is possible. How much memory are you giving to your JVM? You mentioned
that it was low...
> If so, I'd need to find a stacktrace that contains *my* code that has
> this object locked?
Yes. I don't think that all locked objects are displayed in a thread
dump. You'd have to use a debugger to determine that.
> Because I did find one, but I'm not sure what it's telling me. Or can
> anything be told, since the VM threw an OOME and now all bets are
> off?
> - waiting on <0x45251bb0> (a sun.misc.SoftCache)
[snip]
> at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:442)
> at java.util.Date.toString(Date.java:981)
> at java.lang.String.valueOf(String.java:2131)
> at java.lang.StringBuffer.append(StringBuffer.java:370)
> - locked <0x4b94e2e8> (a java.lang.StringBuffer)
> at com.xx.LineItem.setProductionDate(LineItem.java:1181)
[snip]
It looks like your code is waiting on the same lock, but that doesn't
mean that your code is responsible for /obtaining/ the lock. As you can
see, your code is also waiting for that lock.
It turns out that ResourceBundle has a private static SoftCache object
that it uses to store information for the resource bundles that have
been loaded (so that it doesn't have to keep re-loading bundles
requested more than once).
From the 1.4.2_10 source code:
/**
* The cache is a map from cache keys (with bundle base name,
* locale, and class loader) to either a resource bundle
* (if one has been found) or NOT_FOUND (if no bundle has
* been found).
* The cache is a SoftCache, allowing bundles to be
* removed from the cache if they are no longer
* needed. This will also allow the cache keys
* to be reclaimed along with the ClassLoaders
* they reference.
* This variable would be better named "cache", but we keep the old
* name for compatibility with some workarounds for bug 4212439.
*/
private static SoftCache cacheList = new
SoftCache(INITIAL_CACHE_SIZE, CACHE_
LOAD_FACTOR);
Since this thing is private, it is unlikely that you have obtained a
lock on this object that hasn't been released.
I would blame the OOM unless you can prove that deadlock occurred at
some point before that. These look like harmless calls to me. I think
you just might be running out of memory. :(
-chris
signature.asc
Description: OpenPGP digital signature
