On an older cluster like that, rebalances will stop-the-world and kill your throughput. Much better to have a bunch of consumer groups, one per topic, so they can rebalance independently.
On Wed, Feb 26, 2020, 1:05 AM Mark Zang <deepnight...@gmail.com> wrote: > Hi, > > I have a 20 brokers kafka cluster and there are about 50 topics to consume. > > Between creating a consumer for each topic and creating a consumer for all > 50 topics, what is the pros and cons? > > What would be the suggested way if I enable auto commit for each 10 > seconds? > > Kafka client version is 0.10.2 > > > Thanks! > > Mark >
