Hi Gowtham, that output indicates that there is no lag for the consumer group involved. What makes you think there is?
On Fri, Sep 18, 2020 at 7:21 PM Gowtham S <gowtham.co....@gmail.com> wrote: > Thanks for the reply Liam Clarke-Hutchinson, Shaohan Yin > > > Hi, what is the output of kafka-consumer-groups.sh --describe for that > > consumer's group please? > > GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST > CLIENT-ID > > oneg one 0 4838037 4838037 0 fa9d1602 > localhost > fa9d1602 > > > The poll doesn't return always zero if we make 100 poll calls in 100 > seconds it returns 0 records on 70% of poll calls. > > > You could check if there are any throttles on the broker. The > > metric kafka_server_quota_throttle_time might do you a favor. > > > There are no quota throttles. We are using the default quota configuration. > Please let us know if we miss any configuration/metrics to change/listen? > > > With regards, > Gowtham S, MCA > > > > On Fri, 18 Sep 2020 at 09:51, Shaohan Yin <shaohan....@gmail.com> wrote: > > > Hi Gowtham, > > > > You could check if there are any throttles on the broker. The > > metric kafka_server_quota_throttle_time might do you a favor. > > > > Cheers > > > > On Fri, 18 Sep 2020 at 07:20, Liam Clarke-Hutchinson < > > liam.cla...@adscale.co.nz> wrote: > > > > > Hi, what is the output of kafka-consumer-groups.sh --describe for that > > > consumer's group please? > > > > > > On Thu, 17 Sep. 2020, 7:37 pm Gowtham S, <gowtham.co....@gmail.com> > > wrote: > > > > > > > Hello All, > > > > We are consuming a topic with a single partition, the > > consumer.poll(1000) > > > > returns "0" records mostly event if we have more than 10000 records > in > > > lag. > > > > In which case it will behave like this. > > > > We are using Kafka-2.4.0 client and 2.4.0 broker. The single record > > size > > > > is 100Kb. > > > > > > > > Consumer configuration > > > > > > > > - fetch.max.bytes=102428800 > > > > - fetch.min.bytes=1 > > > > - receive.buffer.bytes=-1 > > > > - max.partition.fetch.bytes=102428800 > > > > - max.poll.records=1000 > > > > > > > > Can anyone help us to resolve this? > > > > > > > > Thanks and regards, > > > > Gowtham S, MCA > > > > > > > > > >
