Re: Finding consumer Lag

2020-08-11 Thread Indu V
We are inserting 50,000 records to Kafka from our application. Its a single thread and transactional. Regards, Indu V From: manoj.agraw...@cognizant.com Sent: 12 August 2020 10:51 To: users@kafka.apache.org Subject: Re: Finding consumer Lag [External Email

Re: Finding consumer Lag

2020-08-11 Thread Manoj.Agrawal2
Can you please share what action you are performing and how ? On 8/11/20, 10:19 PM, "Indu V" wrote: [External] Hi Team, I am facing an issue in a clustered Kafka environment, org.apache.kafka.common.KafkaException: Cannot perform send because at least one previous transact

Re: Finding consumer Lag

2020-08-11 Thread Indu V
Hi Team, I am facing an issue in a clustered Kafka environment, org.apache.kafka.common.KafkaException: Cannot perform send because at least one previous transactional or idempotent request has failed with errors. at org.apache.kafka.clients.producer.internals.TransactionManager.failIfNotReadyF

Re: Finding consumer Lag

2020-06-08 Thread Kumar Jayanti
Hi, Did you checkout the following command kafka-consumer-groups.sh --zookeeper zkhost:2181/kafka-cluster --describe --group groupname This will return the LAG information and it is defined as : “the difference between the consumer Current-Offset and the broker Log-End-Offset for this topic partit

Finding consumer Lag

2020-06-08 Thread Indu V
Hi Team, I am facing an issue in finding the lag (unconsumed record count) from Kafka against a topic. I have another process that has to be run after all records are consumed. Each topic has given a single partition with the replication factor as 1. Lag is computed by taking the difference of