Re-posting due to bad formatting (sorry folks) Willy, all,
I am new to the list and let me use the chance and express my gratitude for this wonderful piece of software. Truly amazing. Thank you Willy, thank you all guys. With few folks here we had some learning and already are experiencing quite good results with HAProxy. Wanted to first of all share that during the tests we achieved up to 45,000 requests per second on SSL on a single 1G box (with same setup/hw below). isn't that amazing? :) Also wanted to ask for your opinion or advise on how we can possibly improve the setup further. It really feels like there is something more out there and we could tune up the setup further. Our use case is: - high request per second traffic (very high PPS/packet per second) - HTTPS - hundreds of thousands of requests per second - gigabytes of traffic /per second - currently handled by hardware LoadBalancers --> aim to replace hardware LoadBalancers with HAProxy What do we have currently in HAProxy: Rate: 26,000 HTTPS requests per second, per single HAProxy server CPU idle: 50% System avg load: 8 Software IRQs %: ~10% What would be great to have: - reduced system load - more idle CPU - ability to push more bandwidth or more requests per second - no Software IRQs (or less), possibly less context switches/interrupts Do you think it's possible to further improve current setup software/configuration wise? Lots of details below. Any help or advise is highly appreciated. Thanks, Eduard ---------------------------------------------- OS/HW Debian 7.8 CPU: 2xE5-2630 2.30GHz (24 threads) NIC: Intel i350-AM4 1 GbE Quad Port TRAFFIC FLOW: Traffic -> HAProxy Server Public Interface -> HAProxy Server Private Interface -> Backend Server(s) Private Interface and back BANDWIDTH: 38 MByte/s in, 40MByte/out (eth0 public, eth1 - private) --net/eth0----net/eth1- recv send: recv send 24M 12M: 13M 28M CONFIGURATION and STATS sysctl: http://pastebin.com/raw.php?i=fiaKcA6P haproxy.cfg: http://pastebin.com/raw.php?i=WvsWKfHa /proc/interrupts: http://pastebin.com/raw.php?i=8xc3S0u5 mpstat 1 60: http://pastebin.com/raw.php?i=fD4R5wZR Some more stats using sar and dstat: http://pastebin.com/raw.php?i=x4WBnaGB TCP Stats (avg): Total: 9500 Established: 8150 Closed: 1000 Orphaned: 2 Timewait: 1000 Ports open: 5000 HA-Proxy version 1.5.12 OpenSSL: openssl-1.0.2a Compiled with: export LIBSSLBUILD=/tmp/libsslbuild OpenSSL: ./config --prefix=$LIBSSLBUILD no-shared no-ssl2 no-ssl3 -DOPENSSL_USE_IPV6=0 no-err enable-ec_nistp_64_gcc_128 zlib HAProxy: make TARGET=linux2628 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_TFO=1 ADDINC=-I$LIBSSLBUILD/include ADDLIB="-L$LIBSSLBUILD/lib -ldl"
