Thank you very much Mr. Israel Ekpo. Really appreciate it. We are using the 0.10 version of kafka and in the process of upgrading to 2.6.1 . Planning in process and Yes, these connections to zookeepers are for Kafka functionality.
frequently there are incidents where zookeepers get bombarded with around 8,000 to 10,000 connections from multiple clients. Do you suggest we can configure globalOutstandingLimit to 8000 ? should this be set in zoo.cfg file in Zookeeper ? On Fri, Jul 16, 2021 at 7:11 PM Israel Ekpo <israele...@gmail.com> wrote: > Hello, > > I am assuming you are using Zookeeper because of your Kafka brokers. What > version of Kafka are you using. > > I would like to start by stating that very soon this will no longer be an > issue as the project is taking steps to decouple Kafka from Zookeeper. Take > a look at KIP-500 for additional information. In Kafka 2.8.0 you should be > able to configure the early access version of running Kafka without > Zookeeper. > > When running Kafka in KRaft Mode (Without Zookeeper) you do not need to > worry about this throttling since Zookeeper is not part of the > architecture. > > To answer you questions, some changes were made to prevent Zookeeper from > getting overwhelmed hence to avoid recurring crashes during stressful loads > restrictions can be configured to manage the load on Zookeeper via the > throttling settings > > https://issues.apache.org/jira/plugins/servlet/mobile#issue/ZOOKEEPER-3243 > > You should be able to see additional information in the administration > guides on how this is configured > > https://zookeeper.apache.org/doc/r3.2.2/zookeeperAdmin.html > > For example, the configuration setting below > > globalOutstandingLimit > > (Java system property: *zookeeper.globalOutstandingLimit.*) > > Clients can submit requests faster than ZooKeeper can process them, > especially if there are a lot of clients. To prevent ZooKeeper from running > out of memory due to queued requests, ZooKeeper will throttle clients so > that there is no more than globalOutstandingLimit outstanding requests in > the system. The default limit is 1,000. > > I hope this gets you started > > Feel free to reach out if you have additional questions > > Have a great day > > > On Fri, Jul 16, 2021 at 7:44 AM Kafka Life <lifekafka...@gmail.com> wrote: > > > Dear KAFKA & Zookeeper experts. > > > > 1/ What is zookeeper Throttling ? Is it done at zookeepr ? How is it set > > configured ? > > 2/ Is it helpful ? > > >
