Ok, can anybody grant me access to the Kafka JIRA so I can work on the ticket I'm going to create. My username is "GFriedrich".
Thanks in advance. Kind regards Georg Friedrich -----Original Message----- From: Ryanne Dolan <ryannedo...@gmail.com> Sent: Thursday, March 18, 2021 3:46 PM To: dev <dev@kafka.apache.org> Subject: Re: MirrorMaker 2.0 - Offset Sync - Questions/Improvements I am not certain, but I'd guess a KIP is not required. I think this would be considered a bug fix. Ryanne On Wed, Mar 17, 2021, 5:34 PM Georg Friedrich <georg.friedr...@webfleet.com> wrote: > Hi Ryanne, > > thank you for your response. > > 1) Yes, right - I've missed the condition that is part of the > PartitionState class. Thanks for pointing me. :) > > 2) Ok, how should I go on there? Shall I create a ticket or a KIP or > even both? From my point of view this is not a major change, but for > people relying on the fact that those topics are always created this > may look different. > > Kind regards > Georg Friedrich > > -----Original Message----- > From: Ryanne Dolan <ryannedo...@gmail.com> > Sent: Wednesday, March 17, 2021 4:57 AM > To: dev <dev@kafka.apache.org> > Subject: Re: MirrorMaker 2.0 - Offset Sync - Questions/Improvements > > Georg, sorry for the delay, but hopefully I can still help. > > 1) I think the detail you're missing is that the offset syncs are very > sparse. Normally, you only get a new sync when the Connector first > starts running. You are right that it is possible for a consumer to > lag behind the most recent offset sync, but that will be a rare, transient > condition, e.g. > when the Connector first starts running. > > 2) I think you are right -- disabling checkpoints probably should also > prevent those topics from being created. I'd support that change. > > Ryanne > > On Fri, Feb 26, 2021, 4:24 PM Georg Friedrich < > georg.friedr...@webfleet.com> > wrote: > > > Hi, > > > > recently I've started to look deeper into the code of MirrorMaker > > 2.0 and was faced with some confusing details. Maybe you can point > > me into a right direction here. > > > > > > * The line at > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi > > th > > ub.com%2Fapache%2Fkafka%2Fblob%2F02226fa090513882b9229ac834fd493d71a > > e6 > > d96%2Fconnect%2Fmirror%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkafka%2F > > co > > nnect%2Fmirror%2FOffsetSyncStore.java%23L52&data=04%7C01%7Cgeorg > > .f > > riedrich%40webfleet.com%7Cf53651fc33834e4a793d08d8e8f8c563%7Ce648a63 > > 41 > > 151497c97970f975bddecc0%7C0%7C0%7C637515502445685286%7CUnknown%7CTWF > > pb > > GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M > > n0 > > %3D%7C1000&sdata=D3%2BdZuT3d7wvqLN%2F3q5zIi07iwI7ZO1doxVq2NSHjWU > > %3 > > D&reserved=0 checks whether the offsets that get translated are > > smaller than the last offset sync. > > If this is the case, no translation happens. But I'm confused here: > > Isn't this a potential issue? What if some consumers are slow in > > regards to processing messages from Kafka and fall back behand the > > offset sync process of the MirrorMaker. > > In this case the MirrorMaker would stop to translate any offsets. Do > > I miss something here or is this really broken? > > * I'm wondering: One is able to deactivate emitting checkpoints to > the > > target cluster. But when this happens, the offset sync topic is > > still written to the source cluster. Why is that? As far as I can > > see the only consumer of the offset sync topic is the checkpoint > > connector. So one could also deactivate the whole offset sync > > production entirely when disabling emitting checkpoints. Or is there > > again something that I miss? If not, is this worth a KIP? > > > > Thanks in advance for your answers and help. > > > > Kind regards > > Georg Friedrich > > >
