Peter Rifel wrote:
________________________________________
From: Mark Thomas [ma...@apache.org]
Sent: Thursday, December 26, 2013 3:19 PM
To: Tomcat Users List
Subject: Re: Request Timeout and empty post data issue
On 26/12/2013 20:45, Peter Rifel wrote:
Hello,
I'm currently running Tomcat 7.0.42 on Ubuntu 12.04 with OpenJDK
1.7.0_25. I'm using Apache Tomcat Native library 1.1.27 with APR
version 1.4.6.
I'm noticing in my access logs that some of our POST requests don't
have any POST data and all have response times of a few ms over
20000ms. I'm trying to figure out whether this issue is client side
or server side. The response code and response size for these
requests are normal. Can anyone tell me under what circumstances
this would happen?
It depends.
The connectionTimeout is also the socket timeout. If at any time Tomcat
is (trying to) read data from the client or write data to the client and
no data is available (reading) or read (writing) for the timeout period
then the read / write will fail.
It sounds like the client sends the request and then either doesn't send
the whole body, only sends part of the body or pauses for longer than
connectionTimeout while sending the body. The request body is therefore
missing / incomplete so can't be processed. Whether that results in an
error response will depend on your application.
I noticed that our connector's connectionTimeout
is set to 20000ms, but it wouldn't make sense for that value to be a
part of this issue because a connection timeout only occurs when the
URI hasn't been received by tomcat in that amount of time, which is
clearly not happening here (I was able to confirm this with telnet; a
connection timeout will not write anything to the access logs).
The above statement is incorrect. See the explanation preceding it.
This is what the documentation on the Configuration Reference page says, so if
it is incorrect it should be updated.
The "request" is making it to my servlet (logging confirms this) but
for some reason tomcat doesn't see any request parameters and all of
the response times in our access logs are just above 20 seconds.
Does this mean that my servlet is taking 20 seconds to process the
request?
Unlikely. It almost certainly means Tomcat is waiting 20s for the
request body to turn up.
Is there some other timeout value somewhere that defaults
to 20 seconds?
No.
Is there a way for me to see exactly what is taking
so long?
Wireshark.
Its incredibly hard to debug this because we cant reproduce
this bug on our own without any post data and <1% of our production
traffic is having this issue. To try and gather more info on this, I
added a servlet filter that logs all of our POST request parameters
to the access log and I can confirm that there are no parameters on
these specific requests.
That makes it look very much like a client issue. I'd log the size of
the request body as well. A request body size of > 0 and no POST data
would point to a client issue.
The lack of post data makes me think its a client issue, but that
doesn't explain why all of these requests take ~20 seconds to be
processed.
See above.
Mark
I’m logging the Content-length header and it has expected values, mostly
between 300 and 800 bytes. The requests are coming from a custom flash player
that is being in embedded in users’ browsers that are out of our control, so it
does make it difficult to debug with only 1% of our traffic seeing this
problem. I will run wireshark on tomcat to try and get a better idea of where
the issue is, thank you for the suggestion.
Suggestion: if the custom player is under your control, make sure that the problematic
requests are indeed coming from this player, and not from some (malicious or not) other
source. Perhaps some additional "X-" header in the request, with some time-variable key
that can be checked on the server side ?
You probably don't want to spend a lot of time inspecting thousands of packets if the
reason is a misbehaving robot or monitoring agent or similar.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
