I have multiple map and filter phases in my application dag and though I am generating different keys at different points, the data is still local. Re-partitioning for me here is adding unnecessary network shuffling, I want to minimize it.
-Sameer. On Friday, December 15, 2017, Matthias J. Sax <matth...@confluent.io> wrote: > It's not recommended to write a custom partitioner because it's pretty > difficult to write a correct one. There are many dependencies and you > need deep knowledge of Kafka Streams internals to get it write. > Otherwise, your custom partitioner breaks Kafka Streams. > > That is the reason why it's not documented... > > Not sure so, what you try to achieve in the first place. What do you > mean by > > > I want to make sure that during map phase, the keys > >> produced adhere to the customized partitioner. > > Maybe you achieve what you want differently. > > > -Matthias > > On 12/15/17 1:19 AM, Sameer Kumar wrote: > > Hi, > > > > I want to use the custom partitioner in streams, I couldnt find the same > in > > the documentation. I want to make sure that during map phase, the keys > > produced adhere to the customized partitioner. > > > > -Sameer. > > > >
