Re: Strange Kafka throughput issues

2017-08-30 Thread Boris Sorochkin
Can you please do 2 additional tests: 1. Take SSD based instances and put Kafka storage on SSD. 2. Run storage benchmark with the same block size. What throughput do you receive? What EBS storage type do you use? Remember, EBS network is shared with the instance network so you'd get at most half o

Kafka consumer performance with large network delay

2016-07-15 Thread Boris Sorochkin
Hi All, I have Kafka setup with default settings and relatively large network delay (up to dozens milliseconds) between a broker and consumer. This setup has very poor performance (~30kbps per client) despite of no packet loss on the way (the network is very reliable). After investigation of the tr