Hi huqedi, Thanks for taking a look at this, The *poll* is executed by the task anyway (it's an essential part of the logic) and this KIP only aims to piggyback that request - it will not make any kind of additional requests, thus not adding any performance overhead. Thanks for mentioning the issue with the existing latency metric, I added a mention of it in the KIP's *Rejected Alternatives *section.
Thanks, Elkhan On Wed, 30 Aug 2023 at 14:49, hudeqi <16120...@bjtu.edu.cn> wrote: > Hi Elkhan, > I have also done work similar to the partition replication lag mentioned > in this kip. After going online, I discovered a problem: when > `MirrorSourceTask` executes `poll`, it obtains the LEO of the source topic > through the consumer. This logic may take seconds to jitter. , I think this > may affect the performance of the replication itself. > As for the `replication-latency-ms` metric, it is sometimes inaccurate. > For details, see: > https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-15068 > > best, > hudeqi > > > "Viktor Somogyi-Vass" <viktor.somo...@cloudera.com.INVALID > >写道： > > Hi Elkhan, > > > > I think this is quite a useful improvement. A few questions, suggestions: > > 1. How do you calculate the min, max and avg variants? If I understand > > correctly then the metric itself is partition based > > (where replication-offset-lag is the lag of the replica that is being > > consumed) and these are min, max, avg across replicas? > > 2. You briefly mention replication-latency-ms at the end but I think it'd > > be worth writing a bit more about it, what it does currently, how it is > > calculated and therefore why it doesn't fit. > > > > Thanks, > > Viktor > > > > On Sat, Aug 26, 2023 at 3:49 PM Elxan Eminov <elxanemino...@gmail.com> > > wrote: > > > > > Relatively minor change with a new metric for MM2 > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-971%3A+Expose+replication-offset-lag+MirrorMaker2+metric > > > >
