Hi Greg and Chris, Thank you for your feedbacks!
I added the currently existing workarounds to the “Rejected Alternatives” section, and explained why I still think we need a separate solution. I also added the previous proposal to the rejected alternatives, as I agree with your concerns. I tried to come up with a bit different regex based solution, and modified the KIP based on that. I think it is a more straightforward way to get the desired behavior. Let me know your thoughts. Thanks, Patrik On 2024/07/30 18:57:18 Chris Egerton wrote: > Hi Patrick, > > I share Greg's concerns with the feature as it's currently proposed. I > don't think I could vote for something unless it made replication of > genuinely internal topics and replication cycles impossible, or at least > significantly less likely. > > Best, > > Chris > > On Tue, Jul 30, 2024, 14:51 Greg Harris <gr...@aiven.io.invalid> > wrote: > > > Hi Patrik, > > > > Thanks for the KIP! > > > > Your motivation for this KIP is reasonable, because it is definitely > > possible for the ".internal" suffix to collide with real topics. It would > > have been nice if the original design included some mm2-specific namespace > > like "mm2.internal" to lessen the likelihood of a collision. > > > > However, this is a problem that has numerous existing workarounds: > > * Use a custom ReplicationPolicy and override the methods (for existing > > workloads/mirror makers) > > * Use non-conflicting user topic names (for new user topics) > > * Use the replication.policy.separator to use a non-conflicting separator > > character (for new mirror maker setups) > > > > And the feature as-described has significant risks attached: > > * May allow replication cycles and runaway replication > > * Mirrors internal topics that are unusable on the destination cluster > > > > While these risks can be accounted for if a user is attentive (e.g. when > > they're writing their own ReplicationPolicy) it is not a risk-free > > configuration that composes well with other out-of-the-box configurations. > > For example, someone may expect to take their existing configuration, turn > > on this new option, and expect reasonable behavior, which isn't always > > guaranteed. > > > > If you're still interested in this feature, please reference the existing > > workarounds and include them as rejected alternatives so we can know where > > the existing solutions fall short. > > We'd also have to figure out if and how the risks I mentioned could be > > mitigated. > > > > Thanks, > > Greg > > > > On Tue, Jul 30, 2024 at 5:49 AM Patrik Marton <pmar...@cloudera.com.invalid > > > > > wrote: > > > > > Hi Team, > > > > > > I would like to start a discussion on KIP-1074: Make the replication of > > > internal topics configurable > > > < > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1074%3A+Make+the+replication+of+internal+topics+configurable > > > > > > > > > > The goal of this KIP is to make it easier to replicate topics that seem > > > internal (for example ending in .internal suffix), but are just normal > > > business topics created by the user. > > > > > > I appreciate any feedback and recommendations! > > > > > > Thanks! > > > Patrik > > > > > >
smime.p7s
Description: S/MIME cryptographic signature
