Kafka will batch messages, but if the rate of delivery is too slow it'll fall back to delivering only one message per batch. What is the total throughput per broker?
-Ewen On Fri, Jul 15, 2016 at 5:21 PM, Boris Sorochkin <sborn...@gmail.com> wrote: > Hi All, > I have Kafka setup with default settings and relatively large network delay > (up to dozens milliseconds) between a broker and consumer. This setup has > very poor performance (~30kbps per client) despite of no packet loss on the > way (the network is very reliable). > After investigation of the traffic on wire I discovered, that Kafka mostly > sends messages one per packet, waiting each ack sequentially. In my case > messages in Kafka may be pretty small (800-900 bytes). > > I'm using Kafka version 8.2.2.2 and see the following setting > *channel.socket.setTcpNoDelay(true)* in > *core/src/main/scala/kafka/network/BlockingChannel.scala* > > Summarizing, it seems that Kafka not only disables' Nagle's TCP > optimization but sending small packets not even utilizing whole available > MTU. > > Probably I'm not getting something, so what is the best way to configure > Kafka to utilize network properly? > > Thanks in advance > Regards, > -- Thanks, Ewen
