Producers usually do not query zookeeper at all. Consumers usually query zookeeper at beginning or rebalance. It is supposed to be in frequent if you don¹t have consumers come and go all the time. One exception is that if you are using zookeeper based consumer offset commit, it will commit offset to zookeeper frequently. In Kafka, the most heavily used mechanism for zookeeper is zookeeper listener and they are not fired in a regular frequency.
The limitation of Zookeeper usage for Kafka I am aware of is probably the size of each zNode. As long as you don¹t have so many partitions that zNode cannot handle, it should be fine. Thanks. Jiangjie (Becket) Qin On 4/20/15, 5:58 AM, "Achanta Vamsi Subhash" <achanta.va...@flipkart.com> wrote: >Hi, > >Could anyone help with this? > >Thanks. > >On Sun, Apr 19, 2015 at 12:58 AM, Achanta Vamsi Subhash < >achanta.va...@flipkart.com> wrote: > >> Hi, >> >> How often does Kafka query zookeeper while producing and consuming? >> >> Ex: >> If there is a single partition to which we produce and a HighLevel >> consumer running on it, how many read/write queries to zookeeper happen. >> >> Extending further, multiple topics with ~100 partitions each, how many >> zookeeper calls will be made (read/write). >> >> What is the max limit of no of partitions / kafka cluster that zookeeper >> can handle? >> >> -- >> Regards >> Vamsi Subhash >> > > > >-- >Regards >Vamsi Subhash
