Hello,
Following is an issue we are trying to debug at the moment : Running org.apache.kafka.tools.ProducerPerformance code to performance test the kafka cluster. As a trial cluster has only one broker and zookeeper with 12GB of heap space. Running 6 producers on 3 machines with same transaction id (2 producers on each node). Below are the settings of each producer, kafka-run-class org.apache.kafka.tools.ProducerPerformance --print-metrics --topic perf1 --num-records 9223372036854 --throughput 250000 --record-size 200 --producer-props bootstrap.servers=localhost:9092 buffer.memory=524288000 batch.size=524288 for 2 hours all producers run fine, then suddenly throughput of all producers increase 3 times and 4 producers on 2 nodes crashes with below exceptions, [2018-07-16 14:00:18,744] ERROR Error executing user-provided callback on message for topic-partition perf1-6: (org.apache.kafka.clients. producer.internals.RecordBatch) java.lang.ClassCastException: org.apache.kafka.clients.producer.internals. RecordAccumulator$RecordAppendResult cannot be cast to org.apache.kafka.clients.producer.internals.RecordBatch$Thunk at org.apache.kafka.clients.producer.internals.RecordBatch.done(RecordBatch. java:99) at org.apache.kafka.clients.producer.internals.Sender. completeBatch(Sender.java:312) at org.apache.kafka.clients.producer.internals.Sender. handleProduceResponse(Sender.java:272) at org.apache.kafka.clients.producer.internals.Sender. access$100(Sender.java:57) at org.apache.kafka.clients.producer.internals.Sender$1. onComplete(Sender.java:358) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:278) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134) at java.lang.Thread.run(Thread.java:748) First machine (2 producers) run fine. Need some pointers on this issue. Queires: why the throughput is increasing 3 times after 2 hours of duration ? why the other producers are crashing ? Regards Nerissa
