Thanks for co-operation B.R but i've got to know that aio doesn't work well with linux centos and i already used it once with one of our content servers for serving large-files. I noted that after enabling aio, the svctime reduced but %util increased upto 20% using command iostat -x -d 3
Regarding gzip compression, i am afraid that our content server is used for large videos serving as well as conversion which consumes 80% of cpu resources and gzip compression would overwhelm the cpus. Also trying to figure out that how nginx works with the terms asynchronous and synchronous ? Best Regards. On Wed, Jun 5, 2013 at 11:46 AM, B.R. <[email protected]> wrote: > Hello, > > On Wed, Jun 5, 2013 at 1:56 AM, shahzaib shahzaib > <[email protected]>wrote: > >> Hello, >> >> We're using nginx-1.2.8 to stream large files size 1G but found >> the slow stream with high utilization of harddrive using command "iostat -x >> -d 3" >> >> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s >> avgrq-sz avgqu-sz await svctm %util >> sdc 0.00 444.00 78.67 7.00 18336.00 3608.00 >> 256.16 3.28 39.17 11.02 94.40 >> sdb 0.00 0.00 0.00 0.00 0.00 >> 0.00 0.00 0.00 0.00 0.00 0.00 >> sda 0.00 0.00 0.00 0.00 0.00 >> 0.00 0.00 0.00 0.00 0.00 0.00 >> >> 16G Ram >> 8X CPU E31240 >> HDD SATA >> >> The best way of serving large files is to use asynchronous transfer to > avoid blocking disk requests. > To do so, have a look at the > aio<http://nginx.org/en/docs/http/ngx_http_core_module.html#aio>directive. > The default behavior uses sendfile, which sends file in a > synchronous fashion. > > Note 1: On Linux, aio will work for writing requests only if activated > alone. To activate asynchronous read requests (which seems coherent to your > use case, since you stream files - I understand yo usend them, thus you > read them), you need to activate the > directio<http://nginx.org/en/docs/http/ngx_http_core_module.html#directio>directive. > Refer to the aio documentaiton (link above) for details. > > Note 2: Using directio will automatically disable sendfile. If you don't > use it, you'll need to deactivate sendfile manually. > > You might also wanna have a look at your output_buffers (once again see > the aio directive documentation) to improve large files buffering. > I read around that buffers large enough (>= 4MiB) are usually advised for > big files (splitting them in less parts). Maybe you would like to make some > tests for your specific use case to find the most appropriate values. > The number of those buffers also have an impact: too much buffers consumes > RAM for nothing, whereas too little of them will slow down requests by > creating an artificial bottleneck. > > Following are the main nginx configuration :- >> >> http { >> include mime.types; >> default_type application/octet-stream; >> #tcp_nopush on; >> client_body_buffer_size 128k; >> client_header_buffer_size 128k; >> client_max_body_size 1200M; >> keepalive_timeout 10; >> ignore_invalid_headers on; >> client_header_timeout 3m; >> client_body_timeout 3m; >> send_timeout 3m; >> reset_timedout_connection on; >> #gzip on; >> access_log off; >> include /usr/local/nginx/conf/vhosts/*; >> >> } >> >> >> > By default, Nginx sets gzip to off. Depending on the content you stream > and since you seem to have a nice CPU potential (you didn't provide > information on CPU usage, you'll need to see if you have room for gzip > compression there), you might wanna activate gzip to reduce the amount of > data transferred, thus lowering the transmission time at the cost of a more > sollicitated CPU... if your content provides room for a sensible > compression efficiency (once again, it depends on your specific use case). > No need for it if the benefit is 1% ;o) > >> >> >> Would be very thankful If someone could help me with better configuration >> of nginx to reduce HDD-utilization. >> >> Best Regards.. >> >> _______________________________________________ >> nginx mailing list >> [email protected] >> http://mailman.nginx.org/mailman/listinfo/nginx >> > > No other idea. > > Hope I helped, > --- > *B. R.* > > _______________________________________________ > nginx mailing list > [email protected] > http://mailman.nginx.org/mailman/listinfo/nginx >
_______________________________________________ nginx mailing list [email protected] http://mailman.nginx.org/mailman/listinfo/nginx
