Hello, *Kafka dev community, @matthiasJsax* Can you comment on below question? It is very important for us since we are getting inconsistencies due to current design
On Sun, Jul 9, 2023 at 6:15 PM Pushkar Deole <pdeole2...@gmail.com> wrote: > Hi, > > We have a kafka streams application that consumes from multiple topic with > different keys. Before processing these events in the application, we want > to repartition those events on a single key that will ensure related events > are processed by same application instance. e.g. the events on multiple > topics are related to same call however they are keyed on multiple keys on > those topics and hence go to different application instances but after > consuming those events by the streams, we want to partition them again on > the call id and then process them in that order. Is this possible? > I got to know about through() method on the KStream which seems to be > doing similar thing however not sure if it would achieve the below > functionality: > > Call with id C123 is initiated and following events arrive on 3 topics > with respective keys: > > Event 1 on TopicA: with key a1 > Event 2 on TopicB: with key b1 > Event 3 on TopicC: with key c1 > > Let's assume these are consumed by 3 different instances of kafka streams > application however those application process them with through() method > with a consolidatedTopic with the key C123. > Will this ensure that these 3 events go to the same partition on > consolidatedTopic, and hence after repartitioning, those will be consumed > by same instance of application? >
