Hi
This seems to be quite similar to what I have seen, and others have also
posted about the last couple of years.
I've done a lot of investigation, but have not found a real solution.
My problem is that the threads gets stuck in "W" / "Sending reply" state.
Are you using mod_php ? If so, it might be that one thread is doing
work, while the other threads are waiting to acquire a lock on the php
session file.
When PHP session files are used, the request are processed sequentially,
and not in parallel for a user.
I suggest you also use "strace" unix command to see what the different
threads are doing, that might be easier than using gdb.
When I use strace, I see that one thread is hanging on "poll" system
call, until the timeout (Timeout Apache directive - default 300 seconds)
occur. Then I see that the next thread hangs in "poll" and so on.
So if you set the "Timeout" lower, it might improve your situation.
You should also run "netstat", and see what the status of the sockets
are. If they are almost all in CLOSE_WAIT, I think you are seeing
something similar to me.
Here are some links to similar issues, I'm not sure if it will help you :
http://www.nabble.com/RE:-requests-time-out-under-load,-no-warnings-in-logs-p13065495.html
http://mail-archives.apache.org/mod_mbox/httpd-users/200611.mbox/[EMAIL PROTECTED]
( and the other posts to that thread)
http://www.linuxforums.org/forum/servers/27519-alright-im-dying-here-apache-help-mysql-php.html
http://osdir.com/ml/apache.mod-ssl.user/2004-10/msg00009.html
I have also seen other posts from people with very similar problems,
lots of thread stuck in a state.
What is your environment ? I.e. what version of Apache, what operating
system and version, are you generating dynamic pages, and if so, what
tools (for example php) are you using ?
Do you have a firewall or a load balancer in front of your web server ?
Are almost all the requests from the same IP adresse ? Are almost all
the request for the same URL ?
What my theory is at the moment, after throwing away some other
theories, are that somehow a client creates several hundred requests for
the same URL.
Since in my case this happens fairly often, several times a day on a
uncontroversial high traffic site, from different users, I do not think
it is someone doing a DoS attack.
I think it might be a problem with a firewall or a proxy or maybe the
browser (I have seen it happen for people using both Windows and Mac,
both Firefox and IE), that sits between the web server and the end user.
Somehow the closing of the socket/connection by the browser is not
picked up by the web server. And somehow the browser issues lots of
requests.
I have not completely ruled out that it might be a Apache web server bug
as well.
But since I have no way of reproducing this, apart from observing it
happening quite often, it is very hard to get any further.
So any help would be greatly appreciated.
Regards
Alf Hogemark
Sander Temme wrote:
On Nov 7, 2007, at 9:31 AM, Andrew Rosolino wrote:
I restarted apache and all was fine.. but then 20 minutes later they
went
back all into a reading state.. it appears as if slowly each
processes goes
into the reading state?? I dont understand what the problem is.
If you have gdb on the box (and have some acumen using it), you can
attach to one of the processes and see if it is hanging in something
obvious.
If not, and your traffic is plaintext, I recommend the tcpdump
approach to see what is going on traffic-wise, and what triggers the
issue. I'd dump the traffic to disk (tcpdump -i ethX -s 0 -w
/wherever/traffic.dump port 80) and take it over to a box with
Ethereal/Wireshark for some cozy analysis.
If you're being DOS attacked by trickle requests, you could try
setting a very low timeout (default is 5 minutes which doesn't seem to
be working for you) and perhaps use mod_evasive or somesuch to flag
and firewall the bad clients.
S.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: [EMAIL PROTECTED]
" from the digest: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
