You may want to check first which "keep alive" settings they have for the whole 
chain.
If one of them happened to be close to 15 minutes, that might be a strong hint.

Another tool is "netstat", to see exactly in which TCP state these persistent connections find themselves in. (another bad sentence, just keeping in style here).

Dante Bell wrote:
Hi Chris,

I did indeed read and digest Mark's email and talked to the vendor about
that issue. The stack trace on the old blog post is from the one Mark
was helping out with (man, that was a really bad sentance!).

This is a different issue :( I don't have a stack trace and I don't have
access to the lab they are running these tests in. I've requested the
stack traces when this happens, but haven't received those yet.

Your question about 'crash' is valid and the explanation I received was
that the load test application crashes. That's all I have at this time
from them. I'm helping them from a dark, distant planet and only see the
things they want me to see ;) Weirdly, it doesn't sound like TC is dead
from what they are telling me, after 15 minutes it starts serving up db
responses!

Yes, they are using mod_jk.



On 08/10/2011 12:55 PM, Christopher Schultz wrote:
Dante,

On 8/10/2011 11:57 AM, Dante Bell wrote:
We are seeing that after an application crash (customized load
tester with minimal error handling so it crashes often)
When you say "crash", do you mean you get a stack trace in the logs and
Tomcat stays up, or do you mean that you bring-down the JVM? If you
bring-down the JVM, what is the error that is occurring (check hs_*.txt
files laying around in the working directory for that)?

that TC isn't releasing the connection for about 15 minutes.
If TC is truly dead, then it's not holding connections at all. That
would be the OS holding them.

What makes you think they are not being "released"? What counts as
"released"?

I've reviewed some of the worker directives, but I'm really unsure as
to which one or combination would shorten this interval
significantly.
Does that mean you are using mod_jk/mod_proxy_ajp? Good to have that
kind of information.

The Apache server still serves up static content, which makes me
think that there isn't anything at the OS or Apache layer that is
causing the connection to hang around (granted, this isn't an
absolute and we are investigating these 2 components also).
So you're using Apache httpd, too. Also good to know.

We've done some minor TCP/IP tuning in the Solaris stack, and that
has helped with other issues regarding heavy loads.
On Solaris.

If TC is the culprit, would we need to be setting the advanced
connector directives such as:
|recovery_options        |4: close the connection to Tomcat, if we
detect an error when writing back the answer to the client (browser)
That depends upon what the errors actually are. Care to tell us about
them?

PS. Configs can be found at: http://bit.ly/pFIzO0
Sigh. You should look into "template" workers.

Apache httpd MaxClients setting default is 256. <Connector> MaxThreads
is set to 750, so Tomcat should have almost 3 times more than you need.
Where do you see 750 stuck threads?

I looked at your thread dump. You clearly have not read Mark's previous
response on this list where he told you exactly what was happening: your
webapp is killing itself with these SingleThreadModel servlets. This is
not thread starvation due to configuration, this is thread starvation
due to a poorly-implemented web application.

Apache:* Apache HTTP Server Version 2.2 -- prefork with mpm *Tomcat:*
6.0.20 *JK Connector:* Same as whatever is bundled in with Apache 2.2
(from customer) *Solaris* Solaris 10 10/09 s10s_u8wos_08a SPARC
Aah, here's all the configuration information. Description then context.
Not the best term paper I've ever read. :(

I think you mean "prefork MPM". Apache httpd does not bundle mod_jk.
Check your version.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Reply via email to