Dear Maxim, Thanks for your response, I don't think it will help much since all of my files are larger enough than 2MB
Regarding AIO problem in linux, do you think using AIO + sendfile together on FreeBSD will be better in performance in my case? Respect ============================================ On Mon, Jun 10, 2013 at 3:25 PM, Maxim Dounin <[email protected]> wrote: > Hello! > > On Sun, Jun 09, 2013 at 07:24:49PM +0430, Host DL wrote: > > > Hello Maxim, > > > > Thanks for your response, and sorry that I am new to mailing list and my > > 1st message may was not very clear to you > > > > I've already read all posts in this conversation and all tuning options > has > > been tested > > > > I'm using 8x 2TB SATA ENT in RAID10 level + 64G RAM on my box > > CentOS 5.9 x64_84 / 2.6.18-348.6.1.el5 > > > > nginx.conf: > > > > worker_priority -10; > > worker_processes 64; > > worker_rlimit_nofile 20000; > > > > events { > > worker_connections 2048; > > use epoll; > > worker_aio_requests 128; > > } > > > > http { > > sendfile off; > > tcp_nopush on; > > tcp_nodelay on; > > aio on; > > directio 2m; > > #directio_alignment 4k; > > output_buffers 1 1m; > > > > keepalive_timeout 15; > > > > ...... > > } > > > > During the peak time connections will reach up to 14-15K in total and > more > > than 1Gbit/s outgoing throughput > > Please note that the server was stable with about 12K connections in the > > peak time and about 1-1.1Gbit/s throughput but after adding another VH > with > > about 2-3K connections it seems that server is unable to handle the > request > > properly at the peak time > > Its expected the throughput to exceed the previous ~1.1Gbit/s rate but it > > doesn't, Even it doesn't reach to 1Gbit/s while the connections are now > > getting more and bigger > > > > During every peak time the LA will each to the number of nginx workers ( > 64 > > for my current config ) and will stay at the same rate to the end of peak > > time, all processes are in D state and the interesting thing, memory is > not > > being used fully and it may push about 30-40G with about 20-30% I/O wait > > Main problem with AIO on Linux is that it requires directio to > actually work asynchronously. I would assume you've just reached > a critical number of synchronous requests to disks as due to > "directio 2m" in your config (and that's why you see all workers > in D state). Try tuning directio to a lower value to see if it > helps. > > Note well: memory is not used for filesystem cache with directio, > so there is no surprise it's not being used fully. > > -- > Maxim Dounin > http://nginx.org/en/donation.html > > _______________________________________________ > nginx mailing list > [email protected] > http://mailman.nginx.org/mailman/listinfo/nginx >
_______________________________________________ nginx mailing list [email protected] http://mailman.nginx.org/mailman/listinfo/nginx
