# distribute network IRQs among CPU0-CPU3 IRQ_NET_1=$(grep eth0-rxtx-1 /proc/interrupts | cut -d: -f1 | sed "s/ //g") IRQ_NET_2=$(grep eth0-rxtx-2 /proc/interrupts | cut -d: -f1 | sed "s/ //g") IRQ_NET_3=$(grep eth0-rxtx-3 /proc/interrupts | cut -d: -f1 | sed "s/ //g") echo 02 > /proc/irq/$IRQ_NET_1/smp_affinity echo 04 > /proc/irq/$IRQ_NET_2/smp_affinity echo 08 > /proc/irq/$IRQ_NET_3/smp_affinity
Here you are distributing IRQs among CPU1-CPU3, perhaps you are trying to distribute them among CPU0-CPU2 (keeping CPU3 only for disk): echo 01 > /proc/irq/$IRQ_NET_1/smp_affinity echo 02 > /proc/irq/$IRQ_NET_2/smp_affinity echo 04 > /proc/irq/$IRQ_NET_3/smp_affinity -----Original Message----- From: jtomo...@yahoo.com.INVALID [mailto:jtomo...@yahoo.com.INVALID] Sent: domenica 15 giugno 2014 22:32 To: dev@trafficserver.apache.org Subject: Re: TrafficServer and IO/Wait Hi Baptiste, sorry my late return for your issue. I suggest some environment and software settings, considering 4GB of RAM and 4 CPU threads: 1- check if ATS is linked with libhwloc library (ldd bin/traffic_server | grep libhwloc) if not, recompile using it 2- remove irqbalance (for Ubuntu distro: sudo apt-get purge irqbalance) 3- reserve the last core (CPU3) to disk IRQs. In this case, distribute network IRQs among all cores and set ATS threads to use the first 3 cores (CPU0, CPU1, CPU2). # set disk IRQs to CPU3 for DEV in vmw_pvscsi ata_piix; do for IRQ in $(grep $DEV /proc/interrupts | cut -d: -f1 | sed "s/ //g"); do echo 08 > /proc/irq/$IRQ/smp_affinity done done # distribute network IRQs among CPU0-CPU3 IRQ_NET_1=$(grep eth0-rxtx-1 /proc/interrupts | cut -d: -f1 | sed "s/ //g") IRQ_NET_2=$(grep eth0-rxtx-2 /proc/interrupts | cut -d: -f1 | sed "s/ //g") IRQ_NET_3=$(grep eth0-rxtx-3 /proc/interrupts | cut -d: -f1 | sed "s/ //g") echo 02 > /proc/irq/$IRQ_NET_1/smp_affinity echo 04 > /proc/irq/$IRQ_NET_2/smp_affinity echo 08 > /proc/irq/$IRQ_NET_3/smp_affinity 4- set these values at records.config : CONFIG proxy.config.cache.ram_cache.size INT 3G CONFIG proxy.config.cache.ram_cache_cutoff INT 4M CONFIG proxy.config.exec_thread.limit INT 3 CONFIG proxy.config.cache.threads_per_disk INT 12 CONFIG proxy.config.task_threads INT 6 5- also consider set these disk tweaks : # set noop scheduler for storage disks for DEV in sdb sdc sdd; do echo 1024 > /sys/block/$DEV/queue/nr_requests echo noop > /sys/block/$DEV/queue/scheduler echo 8192 > /sys/block/$DEV/queue/read_ahead_kb done Feel free to post here your iostats -x 1 results after set these changes Cheers Jay Tomolek
