Re: Poll Records Return Zero Records

2020-09-18 Thread Gowtham S
> > Hi Gowtham, that output indicates that there is no lag for the consumer > group involved. What makes you think there is? > Yes, there is no lag now, when I had huge numbers of lags, I saw that most numbers of the poll calls return zero records, so we thought if we can solve this we can able to

Re: Poll Records Return Zero Records

2020-09-18 Thread 子明小友
Interesting! Liam Clarke-Hutchinson 于 2020年9月18日周五 15:23写道: > Hi Gowtham, that output indicates that there is no lag for the consumer > group involved. What makes you think there is? > > On Fri, Sep 18, 2020 at 7:21 PM Gowtham S > wrote: > > > Thanks for the reply Liam Clarke-Hutchinson, Shaoh

Re: Poll Records Return Zero Records

2020-09-18 Thread Liam Clarke-Hutchinson
Hi Gowtham, that output indicates that there is no lag for the consumer group involved. What makes you think there is? On Fri, Sep 18, 2020 at 7:21 PM Gowtham S wrote: > Thanks for the reply Liam Clarke-Hutchinson, Shaohan Yin > > > Hi, what is the output of kafka-consumer-groups.sh --describe

Re: Poll Records Return Zero Records

2020-09-18 Thread Gowtham S
Thanks for the reply Liam Clarke-Hutchinson, Shaohan Yin > Hi, what is the output of kafka-consumer-groups.sh --describe for that > consumer's group please? GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID oneg one04838037 4838037

Re: Poll Records Return Zero Records

2020-09-17 Thread Shaohan Yin
Hi Gowtham, You could check if there are any throttles on the broker. The metric kafka_server_quota_throttle_time might do you a favor. Cheers On Fri, 18 Sep 2020 at 07:20, Liam Clarke-Hutchinson < liam.cla...@adscale.co.nz> wrote: > Hi, what is the output of kafka-consumer-groups.sh --describe

Re: Poll Records Return Zero Records

2020-09-17 Thread Liam Clarke-Hutchinson
Hi, what is the output of kafka-consumer-groups.sh --describe for that consumer's group please? On Thu, 17 Sep. 2020, 7:37 pm Gowtham S, wrote: > Hello All, > We are consuming a topic with a single partition, the consumer.poll(1000) > returns "0" records mostly event if we have more than 1 r

Poll Records Return Zero Records

2020-09-17 Thread Gowtham S
Hello All, We are consuming a topic with a single partition, the consumer.poll(1000) returns "0" records mostly event if we have more than 1 records in lag. In which case it will behave like this. We are using Kafka-2.4.0 client and 2.4.0 broker. The single record size is 100Kb. Consumer conf