Do we really need a whole DSL operator for this? I think the original name for this operator -- `cancelRepartition()` -- is itself a sign that this is not an operation on the stream itself but rather a command/request to whichever operator would have otherwise triggered this repartition.
What about instead adding a new field to the Grouped/Joined/StreamJoined config objects that signals them to skip the repartitioning? The one downside to this specific proposal is that you would then need to specify this for every stateful operation downstream of the key-changing operation. So a better alternative might be to introduce this `skipRepartition` field, or whatever we want to call it, to the config object of the operator that's actually doing the key changing operation which is apparently preserving the partitioning. Imo this would be more "safe" relative to the current proposal, as the user has to explicitly consider whether every key changing operation is indeed preserving the partitioning. Otherwise you could code up a topology with several key changing operations at the beginning which do require repartitioning. Then you get to the end of the topology and insert one final key changing operation that doesn't, assume you can just cancel the repartition, and suddenly you're wondering why your results are all screwed up On Tue, Aug 3, 2021 at 6:02 PM Matthias J. Sax <mj...@apache.org> wrote: > Thanks for the KIP Ivan! > > I think it's a good feature to give advanced users more control, and > allow them to build more efficient application. > > Not sure if I like the proposed named though (the good old "naming > things" discussion :)) > > Did you consider alternatives? What about > > - markAsPartitioned() > - markAsKeyed() > - skipRepartition() > > Not sure if there are other idea on a good name? > > > > -Matthias > > On 6/24/21 7:45 AM, Ivan Ponomarev wrote: > > Hello, > > > > I'd like to start a discussion for KIP-759: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-759%3A+Unneeded+repartition+canceling > > > > > > This is an offshoot of the discussion of KIP-655 for a `distinct` > > operator, which turned out to be a separate proposal. > > > > The proposal is quite trivial, however, we still might consider > > alternatives (see 'Possible Alternatives' section). > > > > Regards, > > > > Ivan >
