Thanks for the reply Liam Clarke-Hutchinson, Shaohan Yin > Hi, what is the output of kafka-consumer-groups.sh --describe for that > consumer's group please?
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID oneg one 0 4838037 4838037 0 fa9d1602 localhost fa9d1602 The poll doesn't return always zero if we make 100 poll calls in 100 seconds it returns 0 records on 70% of poll calls. You could check if there are any throttles on the broker. The > metric kafka_server_quota_throttle_time might do you a favor. There are no quota throttles. We are using the default quota configuration. Please let us know if we miss any configuration/metrics to change/listen? With regards, Gowtham S, MCA On Fri, 18 Sep 2020 at 09:51, Shaohan Yin <shaohan....@gmail.com> wrote: > Hi Gowtham, > > You could check if there are any throttles on the broker. The > metric kafka_server_quota_throttle_time might do you a favor. > > Cheers > > On Fri, 18 Sep 2020 at 07:20, Liam Clarke-Hutchinson < > liam.cla...@adscale.co.nz> wrote: > > > Hi, what is the output of kafka-consumer-groups.sh --describe for that > > consumer's group please? > > > > On Thu, 17 Sep. 2020, 7:37 pm Gowtham S, <gowtham.co....@gmail.com> > wrote: > > > > > Hello All, > > > We are consuming a topic with a single partition, the > consumer.poll(1000) > > > returns "0" records mostly event if we have more than 10000 records in > > lag. > > > In which case it will behave like this. > > > We are using Kafka-2.4.0 client and 2.4.0 broker. The single record > size > > > is 100Kb. > > > > > > Consumer configuration > > > > > > - fetch.max.bytes=102428800 > > > - fetch.min.bytes=1 > > > - receive.buffer.bytes=-1 > > > - max.partition.fetch.bytes=102428800 > > > - max.poll.records=1000 > > > > > > Can anyone help us to resolve this? > > > > > > Thanks and regards, > > > Gowtham S, MCA > > > > > >
