Elkhan, do you want to propose a vote for this KIP or do you have any other ideas to include?
On Tue, Oct 17, 2023 at 2:47 PM Viktor Somogyi-Vass < viktor.somo...@cloudera.com> wrote: > Hi hudeqi, > > Good thinking about the OOM and resource leaks. > The "update.replication.lag.interval.time" I think is almost good but we > should include that it is about a metric (like > "replication.lag.interval.metric.update.time") so it's obvious without the > docs too. > > Thanks, > Viktor > > On Sat, Oct 7, 2023 at 8:53 AM hudeqi <16120...@bjtu.edu.cn> wrote: > >> Hi, Elkhan, Viktor. >> >> I took a look at the updated KIP. I think Viktor mentioned that he did >> not see the relevant configuration, which refers to "(Optional) - >> MirrorConnectorConfig - a configuration to control the poll interval for >> the Consumer.endOffsets() call at LEO acquisition mentioned below". I think >> we can introduce the name of this configuration here, such as >> "update.replication.lag.interval.time", which means that in a separate >> periodic scheduling thread, the lag is calculated by this interval time >> through "consumer.endOffsets - LRO". In addition, for the LRO cache, you >> can add an expired time attribute for each partition. If this expired >> interval time is exceeded before next updated, the LRO of this partition >> can be removed from the cache to avoid possible leaks and OOM. >> >> best, >> hudeqi > >
