We are witnessing a potential bottleneck using piped logging. Each web server 
handles relative large amount of concurrent connections. It is running on 
CentOS 5.x, using worker MPM. We use piped log to cronolog. We are discovering 
when the web server reach ~6K concurrent connections (our request are small in 
nature), we are experiencing logging bottleneck. This is confirmed by looking 
at the Apache Status where the vast majority of the threads are in Logging 
state. This is not an issue that caused by slow disks (almost no I/O wait).

I suspect the issue here is because the pipe buffer size is fixed at 4K (8 x 
512), and with many threads/process potentially writing to that, its write 
became blocked until cronolog get its timeslice to clear out the buffer, but it 
will never catch up since as soon as it is cleared, it is almost immediately 
filled again. We can "patch" the kernel to adjust the pipe buffer size, but I 
would like to avoid if possible.

We can potentially alleviate the issue slightly by specifying piped log in each 
of the virtual host (although all writing back to the same file through pipe to 
cronolog). Given we have very limited number of vhost, and different logging 
can not be specified more detail than vhost/system wide level, this does not 
provide much more capacity.

Does anyone else have similar experience? Did anyone try some of the Apache 
module with logrotate capability?

Note, we can not use a cronjob to rotate the logs then do a graceful restart of 
Apache.

Thanks.



      

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Reply via email to