I am not certain, but I'd guess a KIP is not required. I think this would be considered a bug fix.
Ryanne On Wed, Mar 17, 2021, 5:34 PM Georg Friedrich <georg.friedr...@webfleet.com> wrote: > Hi Ryanne, > > thank you for your response. > > 1) Yes, right - I've missed the condition that is part of the > PartitionState class. Thanks for pointing me. :) > > 2) Ok, how should I go on there? Shall I create a ticket or a KIP or even > both? From my point of view this is not a major change, but for people > relying on the fact that those topics are always created this may look > different. > > Kind regards > Georg Friedrich > > -----Original Message----- > From: Ryanne Dolan <ryannedo...@gmail.com> > Sent: Wednesday, March 17, 2021 4:57 AM > To: dev <dev@kafka.apache.org> > Subject: Re: MirrorMaker 2.0 - Offset Sync - Questions/Improvements > > Georg, sorry for the delay, but hopefully I can still help. > > 1) I think the detail you're missing is that the offset syncs are very > sparse. Normally, you only get a new sync when the Connector first starts > running. You are right that it is possible for a consumer to lag behind the > most recent offset sync, but that will be a rare, transient condition, e.g. > when the Connector first starts running. > > 2) I think you are right -- disabling checkpoints probably should also > prevent those topics from being created. I'd support that change. > > Ryanne > > On Fri, Feb 26, 2021, 4:24 PM Georg Friedrich < > georg.friedr...@webfleet.com> > wrote: > > > Hi, > > > > recently I've started to look deeper into the code of MirrorMaker 2.0 > > and was faced with some confusing details. Maybe you can point me into > > a right direction here. > > > > > > * The line at > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > > ub.com%2Fapache%2Fkafka%2Fblob%2F02226fa090513882b9229ac834fd493d71ae6 > > d96%2Fconnect%2Fmirror%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkafka%2Fco > > nnect%2Fmirror%2FOffsetSyncStore.java%23L52&data=04%7C01%7Cgeorg.f > > riedrich%40webfleet.com%7Cf53651fc33834e4a793d08d8e8f8c563%7Ce648a6341 > > 151497c97970f975bddecc0%7C0%7C0%7C637515502445685286%7CUnknown%7CTWFpb > > GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0 > > %3D%7C1000&sdata=D3%2BdZuT3d7wvqLN%2F3q5zIi07iwI7ZO1doxVq2NSHjWU%3 > > D&reserved=0 checks whether the offsets that get translated are > > smaller than the last offset sync. > > If this is the case, no translation happens. But I'm confused here: > > Isn't this a potential issue? What if some consumers are slow in > > regards to processing messages from Kafka and fall back behand the > > offset sync process of the MirrorMaker. > > In this case the MirrorMaker would stop to translate any offsets. Do I > > miss something here or is this really broken? > > * I'm wondering: One is able to deactivate emitting checkpoints to > the > > target cluster. But when this happens, the offset sync topic is still > > written to the source cluster. Why is that? As far as I can see the > > only consumer of the offset sync topic is the checkpoint connector. So > > one could also deactivate the whole offset sync production entirely > > when disabling emitting checkpoints. Or is there again something that > > I miss? If not, is this worth a KIP? > > > > Thanks in advance for your answers and help. > > > > Kind regards > > Georg Friedrich > > >
