Hello, I am a new kafka user, trying to reach kafka limits :1 000 000 messages per second.
Therefore, I loaded three Azure VMs having this configuration: 16 CPU Cores 56 Go Ram All disks are premium LRS and minimum capacity 512 Go. Download speed 2696.28 Mbit/s Upload speed 1121.13 Mbit/s I am using 0.10.0 kafka version, this is the embedded version in ambari 2.4.2 (hortonworks) Each VM is a broker. I tried the kafka-producer-perf-test.sh script to perform performance test, i used several parameters but I couldn't exceed 210K/s: Number of partitions: 1, 2, 3 per topic Batch size: 4096 , 8192, 16384 Message size: 1024 , 512, 100 Thread number: 1, 2 , 4 , 16 These are different combination of parameters that I used. Any help will be appreciated. Thanks
