Hi all, I have been trying out kafka benchmarks described in Jay's benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines <https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machine>. I'm able to get similar results on a 4-node GbE network whose in-bytes could be saturated at 120MB/s. However, on a 4-node, 10GbE network, I can not get in-bytes higher than 150MB/s. *Has anyone benchmarked kafka on a 10GbE network ? Any rule of thumb on 10GbE network for configurations of broker, producer and consumer ? *
My kafka version is 0.8.1.1 and I've created a topic with 8 partitions with 1 replica distributed evenly among the 4 nodes. Message size is 100 bytes. I use all the default kafka settings. My cluster has 4 nodes, where each node has 32 cores, 128MB RAM and 3 disks for kafka. I've tried increasing message size to 1000 bytes which improved producer's throughput but not consumer's. Thanks, Manu
