Hey Tim, I think the functionality you need is groupByKey() which avoids repartitioning, feel free to check it out here: https://docs.confluent.io/current/streams/developer-guide/dsl-api.html#aggregating. Recommend you to read the whole thing but feel free just to search `groupByKey`.
On Fri, Aug 9, 2019 at 7:14 AM Tim Ward <tim.w...@origamienergy.com> wrote: > I've got an input topic which is keyed by "parent ID". Each message > contains multiple items of data, each for a different "child ID". > > To process these items separately I flatMapValues() the stream to make a > new stream of the inner items of data, keyed by "child ID". > > Now, because I've changed the key, Kafka Streams thinks a repartition is > needed. But in fact it isn't, because all the inner items for a particular > "child ID" will be contained within messages keyed with the same "parent > ID". > > How do I tell Kafka Streams that there is no need to repartition in this > case, because all the data that should remain together in the same instance > of the application will do so without repartitioning? (I appreciate that > Streams can't know about the parent-child relationship unless I *do* tell > it in some way.) > > Tim Ward > > This email is from Origami Energy Limited. The contents of this email and > any attachment are confidential to the intended recipient(s). If you are > not an intended recipient: (i) do not use, disclose, distribute, copy or > publish this email or its contents; (ii) please contact Origami Energy > Limited immediately; and then (iii) delete this email. For more > information, our privacy policy is available here: > https://origamienergy.com/privacy-policy/. Origami Energy Limited > (company number 8619644) is a company registered in England with its > registered office at Ashcombe Court, Woolsack Way, Godalming, GU7 1LQ. >
