I think you will find this article https://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/ useful
On Tue, Nov 14, 2017 at 6:49 PM, Viktor Somogyi <viktorsomo...@gmail.com> wrote: > Hi Jeff, > > I think it's also worth considering that 1K consumer (implying that you > have at least 1 consumer per group) for 1 partition means 1K TCP connection > which means that they have to share the available bandwidth. > Why do you have so many consumer groups? Do you basically want to > multicast? > > Viktor > > > On Tue, Nov 14, 2017 at 4:18 PM, Arunkumar <pm_arunku...@yahoo.com. > invalid> > wrote: > > > Hi Jeff > > Number of partition depends on number of consumers on that particular > > consumer group. So you may have to create your partitions based on that. > > > > ThanksArunkumar Pichaimuthu, PMP > > > > On Monday, November 13, 2017, 5:25:35 PM CST, Jeff Widman < > > j...@jeffwidman.com> wrote: > > > > We're considering an architecture that would result in 5K-10K consumer > > groups consuming from a single topic that has one partition. > > > > What are the reasonable limits for the max number of consumer groups per > > partition and per broker? > > > > Can a single broker be the group coordinator for 1K+ consumer groups? > > > > -- > > > > *Jeff Widman* > > jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) > > <>< > > > > >
