> > Hi Gowtham, that output indicates that there is no lag for the consumer > group involved. What makes you think there is? >
Yes, there is no lag now, when I had huge numbers of lags, I saw that most numbers of the poll calls return zero records, so we thought if we can solve this we can able to increase consumer throughput. With regards, Gowtham S, MCA On Fri, 18 Sep 2020 at 14:07, 子明小友 <xianshengzim...@gmail.com> wrote: > Interesting! > > Liam Clarke-Hutchinson <liam.cla...@adscale.co.nz> 于 2020年9月18日周五 15:23写道： > > > Hi Gowtham, that output indicates that there is no lag for the consumer > > group involved. What makes you think there is? > > > > On Fri, Sep 18, 2020 at 7:21 PM Gowtham S <gowtham.co....@gmail.com> > > wrote: > > > > > Thanks for the reply Liam Clarke-Hutchinson, Shaohan Yin > > > > > > > Hi, what is the output of kafka-consumer-groups.sh --describe for > that > > > > consumer's group please? > > > > > > GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID > > HOST > > > CLIENT-ID > > > > > > oneg one 0 4838037 4838037 0 fa9d1602 > > > localhost > > > fa9d1602 > > > > > > > > > The poll doesn't return always zero if we make 100 poll calls in 100 > > > seconds it returns 0 records on 70% of poll calls. > > > > > > > > > You could check if there are any throttles on the broker. The > > > > metric kafka_server_quota_throttle_time might do you a favor. > > > > > > > > > There are no quota throttles. We are using the default quota > > configuration. > > > Please let us know if we miss any configuration/metrics to > change/listen? > > > > > > > > > With regards, > > > Gowtham S, MCA > > > > > > > > > > > > On Fri, 18 Sep 2020 at 09:51, Shaohan Yin <shaohan....@gmail.com> > wrote: > > > > > > > Hi Gowtham, > > > > > > > > You could check if there are any throttles on the broker. The > > > > metric kafka_server_quota_throttle_time might do you a favor. > > > > > > > > Cheers > > > > > > > > On Fri, 18 Sep 2020 at 07:20, Liam Clarke-Hutchinson < > > > > liam.cla...@adscale.co.nz> wrote: > > > > > > > > > Hi, what is the output of kafka-consumer-groups.sh --describe for > > that > > > > > consumer's group please? > > > > > > > > > > On Thu, 17 Sep. 2020, 7:37 pm Gowtham S, <gowtham.co....@gmail.com > > > > > > wrote: > > > > > > > > > > > Hello All, > > > > > > We are consuming a topic with a single partition, the > > > > consumer.poll(1000) > > > > > > returns "0" records mostly event if we have more than 10000 > records > > > in > > > > > lag. > > > > > > In which case it will behave like this. > > > > > > We are using Kafka-2.4.0 client and 2.4.0 broker. The single > > record > > > > size > > > > > > is 100Kb. > > > > > > > > > > > > Consumer configuration > > > > > > > > > > > > - fetch.max.bytes=102428800 > > > > > > - fetch.min.bytes=1 > > > > > > - receive.buffer.bytes=-1 > > > > > > - max.partition.fetch.bytes=102428800 > > > > > > - max.poll.records=1000 > > > > > > > > > > > > Can anyone help us to resolve this? > > > > > > > > > > > > Thanks and regards, > > > > > > Gowtham S, MCA > > > > > > > > > > > > > > > > > > > > >
