If I understood your question correctly - you are looking for recommended guideline for #partitions / consumer. Kafka doesn’t hold any responsibility for that - and I’m happy to be corrected.
There are two partition assignment strategy - default and sticky. But you should always aim for x/N partitions per consumer (in a given consumer group). x = number of consumers N = number of partitions in a given topic. This concept is irrelevant of Kafka. In any distributed ecosystem you should maintain the above number such that 1) there’s no race between threads I.e. processing same task 2) tasks are evenly distributed amongst workers - unless a worker dies/ exits their pool. So if you have not planned for an even distribution of partitions per consumer thread, you may need to revisit how you have planned to spawn your consumers in your cgroup. I hope that makes sense. Thanks, On Sun, 11 Nov 2018 at 15:29, Sachit Murarka <connectsac...@gmail.com> wrote: > Hi, > > Thanks for reply. > You have shared the command. I am looking for the formula or way to > calculate the number of partitions for my topic. > > Regards, > Sachit > > On Sun, 11 Nov 2018, 20:08 花儿向阳开 <2468946...@qq.com wrote: > > > Hi, > > Maybe you can use the command like: > > kafka-topic.sh --create --topic <topic_name> --partitions > > <partition_number> --replication-factor <replica_number> --zookeeper > > <zookeeperIP:Port> > > I hope it helps! > > > > > > Regards! > > Cathy > > ------------------ 原始邮件 ------------------ > > 发件人: "Sachit Murarka"<connectsac...@gmail.com>; > > 发送时间: 2018年11月11日(星期天) 晚上10:48 > > 收件人: "users"<users@kafka.apache.org>; > > > > 主题: Deciding number of partitions > > > > > > > > Hello All, > > > > I am new to kafka. Could anyone please tell how to decided number of > > partitions for our topic? > > > > > > Kind Regards, > > Sachit Murarka >
