I second Dave's suggestion.
With regards to the consumers round-robining between topics, they usually
round-robin in batches. So you'll probably see a consumer work on a large
batch of records from TopicA before moving on to TopicB. Depending on the
behavior of the producers this might appear the
I would go with #1:
1. It will be easier to add new "batch producers" since you won't need to worry
about re-partitioning
2. You have more control over the parallelism since you can have different
numbers of partitions for each topic
3. You can easily split out your consumer into N consumers if
Hi all,
He is my usecase:
I have three message producers that submits batch of messages to Kafka
from time to time. Let's assume now that one of them just submitted 1k
messages, second one submitted some number of messages after that and
third one also submitted some messages.
I would like
