Hi, I am using kafka producer and broker for a production setup. The expected producer output is 20MBps but I am only getting max of 8MBps. I have verified that we are losing packets by directly connecting to the data source through TCP though the metrics is not reflecting any loss. I went through the performance page where it can reach a speed of 50MBps. Please look at the config and suggest if there is some configuration improvement i can do.
*** *Message Size* *** Message size = 3KB ***** Producer Config **** producer.type = async queue.buffering.max.ms = 100 queue.buffering.max.messages = 4000 request.timeout.ms = 30000 batch.num.messages = 200 **** Broker Config* *** num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=1048576 socket.receive.buffer.bytes=2097152 socket.request.max.bytes=104857600 log.dir=/data1/kafka/logs num.partitions=1 log.flush.interval.messages=1000 log.flush.interval.ms=300 log.retention.hours=48 log.retention.bytes=107374182400 log.segment.bytes=536870912 log.cleanup.interval.mins=1 zookeeper.connect=dare-msgq00:2181,dare-msgq01:2181,dare-msgq02:2181 zookeeper.connection.timeout.ms=1000000 -- Abhinav Anand
