Hi all,

I have been trying out kafka benchmarks described in Jay's
benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
<https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machine>.
I'm able to get similar results on a 4-node GbE network whose in-bytes
could be saturated at 120MB/s. However, on a 4-node, 10GbE network, I can
not get in-bytes higher than 150MB/s. *Has anyone benchmarked kafka on a
10GbE network ? Any rule of thumb on 10GbE network for configurations of
broker, producer and consumer ? *

My kafka version is 0.8.1.1 and I've created a topic with 8 partitions with
1 replica distributed evenly among the 4 nodes. Message size is 100 bytes.
I use all the default kafka settings.
My cluster has 4 nodes, where each node has 32 cores, 128MB RAM and 3 disks
for kafka.

I've tried increasing message size to 1000 bytes which improved producer's
throughput but not consumer's.


Thanks,
Manu

Reply via email to