Hi Akshay Burrow is a popular open-source tool for monitoring the consumer lag in Apache Kafka. However, Burrow depends on ZooKeeper to access the Kafka metadata and track the consumer lag. Therefore, it is not possible to use Burrow without ZooKeeper.
Since you are using Kafka version 3.3.1, you can leverage Kafka's built-in consumer lag monitoring functionality without the need for any additional tools or dependencies. Kafka exposes consumer lag metrics through JMX, and you can use any monitoring system that supports JMX to collect and visualize these metrics. Here are the steps to monitor consumer lag in Kafka: 1. Enable JMX on Kafka brokers and consumers by setting the following JVM system properties: -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Djava.rmi.server.hostname=<hostname> 1. Replace <hostname> with the hostname or IP address of the Kafka broker or consumer machine. 2. Start a monitoring system that supports JMX, such as Prometheus, Grafana, or Datadog. 3. Configure the monitoring system to collect Kafka consumer lag metrics using JMX. For example, if you are using Prometheus, you can use the jmx_exporter tool to expose the Kafka JMX metrics as Prometheus metrics. 4. Visualize the Kafka consumer lag metrics in the monitoring system dashboard. You can create custom dashboards that show the consumer lag for different topics, consumers, and consumer groups. You can also set up alerts to notify you when the consumer lag exceeds a certain threshold. By using Kafka's built-in JMX metrics and a monitoring system that supports JMX, you can monitor consumer lag without the need for additional tools or dependencies. This approach also provides a more lightweight and scalable solution compared to using external tools like Burrow. Thanks Santhosh Gopal Advisor - Architecture,Performance Engineering On Tue, May 9, 2023 at 2:14 PM Akshay Kumar <akshayku...@ameyo.com.invalid> wrote: > Hello team, > > I am using Zookeeper less Kafka (Kafka Kraft - version 3.3.1). I wanted to > monitor consumer lag, so I was using Burrow for that, but I am unable to > use Burrow without Zookeeper. > > Does Burror work without Zookeeper? > > Or what is the better or best way to monitor consumer lag and lag history > for multiple topics, consumers, and consumer groups? > > All the topics are consumed by different-different consumers(applications). > > [image: https://exotel.com/] > Akshay Kumar > Member of Technical Staff - 2 | Exotel > <https://www.exotel.com/?utm_source=global-db&utm_medium=email&utm_campaign=email-signature&utm_term=dg-int&utm_content=shiva-signature> > [image: Facebook] <https://www.facebook.com/Exotel/> [image: Twitter] > <https://twitter.com/Exotel> [image: Linkedin] > <https://www.linkedin.com/company/exotel-techcom-private-limited/> > Ph: +918556063696 > Latest from the Exotel Blog > <https://exotel.com/blog/how-does-an-omnichannel-approach-impact-customer-service/> > > [image: https://exotel.com/] <https://exotel.com/> > *CONFIDENTIALITY NOTE:* This e-mail is intended only for the person or > entity to which it is addressed and contains information that is > privileged, confidential, or otherwise protected from disclosure. > Dissemination, distribution, or copying of this e-mail or the information > contained herein by anyone other than the intended recipient, or an > employee or agent responsible for delivering the message to the intended > recipient, is prohibited. If you have received this e-mail in error, please > delete this message and immediately notify the sender by e-mail. > *NOTE:* This e-mail does not constitute an electronic signature and the > sender does not intend to enter into any agreement by way of this e-mail, > unless otherwise expressly provided by the sender within this e-mail. >
