Hey Valibhav, With only one partition, all of the consumers will end up hitting a single broker (since partitions cannot be split). Whether it is possible to get that number of consumers on a single broker may depend on the message load through the topic. I think there has been some interest in allowing consumers to read from partition replicas and that would help, but I'm not sure if there is anything solid. If the single broker is not enough to handle the load, then you might also consider replicating the topic to several other topics in the same cluster (e.g. using mirror maker), and that would allow you to distribute the consumer load across those other topics. Others might have better ideas.
-Jason On Sun, Aug 2, 2015 at 10:09 PM, Vaibhav Kirte <vaibhav.ki...@sokrati.com> wrote: > Hi, > > I need to know how many consumers can subscribe to a single topic ( with > one partition ). > > I have a requirement such that, > 1. The producer will post to 1 topic having 1 partition. > 1. *20,000-30,000 consumers *should to be able to consume messages. > 2. All of the consumers should receive all messages that are produced. > > will this be possible using kafka ? > is there a limit on number of consumers ? > what will be the number of machines that I will need to satisfy the > requirements ? > > -- > Regards, > Vaibhav Kirte >
