Thankfully when I pulled up jvisualvm on the server and issued thread dumps,
even though the stacktraces for the threads did not come up within jvisualvm
the thread stacktraces were dumped to stdout. So I did get thread
stacktraces in my tomcat log. I won't post entire stacktraces here but
almost all blocked threads seem to be stuck blocking here:
- waiting to lock <0x00002aaaced306f8> (a
org.apache.commons.dbcp.datasources.SharedPoolDataSource)
One of my threads in runnable state has that locked
- locked <0x00002aaaced306f8> (a
org.apache.commons.dbcp.datasources.SharedPoolDataSource)
So I'm pretty sure I've found the thread hanging everything up.... now to
take a look at the code and figure out why lock is not being released.
Highly recommend trying out jvisualvm (it's in your jdk bin) if you're
running 1.6.
