Hi Tom, Thanks for the feedback.
Yes unfortunately it's not possible to automatically detect such misconfiguration. In practice, I expect an administrator to quickly identify if no checkpoints are being emitted and notice the offset-syncs topic is missing or that there's a copy in both cluster. Yes that's right, there is no migration steps. Users will need to stop their connectors, clear the offset-syncs topic and restart the connectors with the new configuration. I've updated the KIP with this explanation. Thanks On Thu, Apr 15, 2021 at 6:39 PM Tom Bentley <tbent...@redhat.com> wrote: > > Hi Mickael, > > In Connect distributed mode the KIP says "it's necessary to set the config > on both the Source and Checkpoint connectors". There's no way to detect > this misconfiguration, because connect just sees two distinct connections. > So if the user gets this wrong then the source connector would try to > create it in the "wrong" cluster, the source task will populate it there, > but the checkpoint task would look in the wrong cluster, and depending on > how allow.auto.create.topics is configured it could create the topic by > side effect. Thus am I right that such a misconfiguration could go > undetected indefinitely? > > It would be good to add to the migration section how people would migrate > from offsets sync topic in the source to the target cluster, or the > reverse. I think it's just a matter of changing the configs and deleting > the old offset-sync topic, right? But, assuming this wouldn't be covered in > the docs, it would be helpful to document it somewhere. > > Thanks, > > Tom > > On Thu, Mar 18, 2021 at 3:51 PM Ryanne Dolan <ryannedo...@gmail.com> wrote: > > > Looks good to me, I'll vote +1. > > > > Ryanne > > > > On Thu, Mar 18, 2021, 10:36 AM Mickael Maison <mickael.mai...@gmail.com> > > wrote: > > > > > Hi, > > > > > > I've not seen any feedback on this KIP. > > > As it's relatively straight forward, I'll open a vote in the next few > > > days if I don't see anything. > > > > > > Thanks > > > > > > On Fri, Feb 26, 2021 at 6:20 PM Mickael Maison <mickael.mai...@gmail.com > > > > > > wrote: > > > > > > > > Hi, > > > > > > > > I would like to start a discussion on KIP-716. This will allow > > > > configuring where MirrorMaker2 creates the offset syncs topic. > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-716%3A+Allow+configuring+the+location+of+the+offset-syncs+topic+with+MirrorMaker2 > > > > > > > > I look forward to your feedback > > > > Thanks > > > > >
