Hi everyone!
Thanks again for all your feedback! The discussion thread has been silent
for a week now, I'll wait a bit for any new feedback and will start the
voting in a couple of days.
Best,
Patrik
On Wed, Aug 21, 2024 at 10:28 AM Patrik Marton wrote:
> Hi Omnia,
>
> Thanks, I updated the KI
Hi Omnia,
Thanks, I updated the KIP with the DefaultTopicFilter changes, to reflect
the new mm2.*.internal pattern.
Best,
Patrik
On Mon, Aug 19, 2024 at 2:24 PM Omnia Ibrahim
wrote:
> Thanks Patrik for updating the KIP.
> One last feedback, should we update `TOPICS_EXCLUDE_DEFAULT` also to
> r
Thanks Patrik for updating the KIP.
One last feedback, should we update `TOPICS_EXCLUDE_DEFAULT` also to reflect
the same pattern instead of `.*[\\-\\.]internal`?
Best,
Omnia
> On 16 Aug 2024, at 13:31, Patrik Marton wrote:
>
> Hi All,
>
> I updated the KIP with the new proposal, and change
Hi All,
I updated the KIP with the new proposal, and changed the name to better
describe the latest proposal and the goal.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1074%3A+Allow+the+replication+of+user+internal+topics
Let me know your thoughts!
Thanks!
Patrik
On Wed, Aug 14, 2024 a
Hi Omina,
We have not considered this path yet that you proposed and I like it as
well and I would also be in favor of a solution that doesn't try to
introduce a new config. Since this KIP would only make it into 4.0 anyway
and your proposal seems like a smaller compatibility break that can
be doc
Hi Patrick,
> One thing that comes up for me is whether we should update the TopicFilter
> too to have this more specific pattern, like "mm2-.*.internal". Ideally I'd
> say yes, but what if a user already has existing .internal user topics and
> relies on the ReplicationPolicy and TopicFilter not
Hi All,
Thanks again for your feedback!
Regarding the boolean solution, yes this would technically let the customer
accidentally replicate "__.*" topics, but only if they modify the
TopicFilter to allow their replication, as by default topics matching this
pattern are excluded. But in this case w
Hi
Thinking of it more, I feel leaving customers to mirroring Kafka’s internal
topics `__.*` either by mistake or intently will be eating up resources from
both mm2 and the clusters for no actual value as the data is not that useful
outside the context of the original cluster.
These topics usua
Hi All,
I have a couple of notes to add
OI-1.Regarding the proposed default regex in the KIP,
"__.*|\\..*|.*-internal|.*\\.internal", when users wish to change this,
wouldn’t they mostly need to keep the __.* part? Especially since __ is
primarily reserved for broker internal metadata topics
Hi all,
1. Yea, thinking of it more, I'm also concerned about the upgrade path and
I think that it may not be worth adding a ton of code to work around that
one time upgrade. It likely needs extra configs as well or documentation to
instruct users, so it may not be worth the price. So let's add th
Hi All,
Thanks again for your feedback!
My thoughts on Viktor's ideas:
1. This would definitely be a clean solution in the end, but I also feel
that the complexity of this solution might outweigh the benefits. If you
guys think, we can give it a thought and figure out what exactly needs to
be ch
Hi Viktor,
Nice to hear from you! My thoughts:
1. I initially liked this idea, but it doesn't seem like the costs to all
existing users outweigh the benefits that would be reaped by just a few.
This is possibly because I'm assuming the upgrade path would be pretty
gnarly, though; if we can make i
Hi all,
I think there are a couple of other alternatives to discuss.
1. We may just simply change the internal topic naming to something like
"mm2internal" and provide upgrade functionality as well. In this scenario
we wouldn't need to tinker with the prefixes or the naming patterns, we
just simp
Hi Patrik,
I think I owe you an apology. After mulling this over for a few more hours
and going over the code base some more, it seems like your initial approach
was actually better in some ways.
With the regex-based approach, it becomes possible for users to configure
the DefaultReplicationPolic
Hi Greg and Chris,
Thank you for your feedbacks!
I added the currently existing workarounds to the “Rejected Alternatives”
section, and explained why I still think we need a separate solution. I also
added the previous proposal to the rejected alternatives, as I agree with your
concerns.
I tri
Hi Patrick,
I share Greg's concerns with the feature as it's currently proposed. I
don't think I could vote for something unless it made replication of
genuinely internal topics and replication cycles impossible, or at least
significantly less likely.
Best,
Chris
On Tue, Jul 30, 2024, 14:51 Gre
Hi Patrik,
Thanks for the KIP!
Your motivation for this KIP is reasonable, because it is definitely
possible for the ".internal" suffix to collide with real topics. It would
have been nice if the original design included some mm2-specific namespace
like "mm2.internal" to lessen the likelihood of
17 matches
Mail list logo
