Re: Kafka streams stores key in multiple state store instances

2024-05-16 Thread Matthias J. Sax
Hello Kay, What you describe is "by design" -- unfortunately. The problem is, that when we build the `Topology` we don't know the partition count of the input topics, and thus, StreamsBuilder cannot insert a repartition topic for this case (we always assume that the partition count is the sam

Kafka streams stores key in multiple state store instances

2024-05-16 Thread Kay Hannay
Hi, we have a Kafka streams application which merges (merge, groupByKey, aggretgate) a few topics into one topic. The application is stateful, of course. There are currently six instances of the application running in parallel. We had an issue where one new Topic for aggregation did have anoth