Take a look at the built in producer performance tools that you can do your own benchmarking with your dataset. If compression is an option consider that as well.
Here's a helpful cheatsheet to use: https://github.com/JNachtwey/kafka-cheat-sheets/blob/master/kafka-performance.md On Fri, Sep 7, 2018, 8:16 AM SenthilKumar K <senthilec...@gmail.com> wrote: > Hello Experts, We are planning to use Kafka for large message set ( size > various from 2 MB to 4 MB per event ). By setting message.max.bytes to 64 > MB value this Kafka Producer allows large messages. But how does it > impacts performance (both producer and consumer)? > > Would like to understand the performance impact of different message size > . Example : producing 50KB vs 1MB . > > What is the optimal message size can be used in Kafka Producer ? > > --Senthil >
