Kafka Streams shared state base on key out-of-the-box and exploit horizontal scaling.
This record redistribution happens automatically based on the key if required. You don't need to explicitly declare it. 1) if you change the key, AND 2) apply a key-based operation (like groupBy or join) afterwards Check out the developer guide for more details: https://docs.confluent.io/current/streams/developer-guide.html#kafka-streams-dsl Look for "data re-partitioning" in the DSL section. Each operator has a statement about it. -Matthias On 10/19/17 6:36 PM, Kristopher Kane wrote: > Storm has the ability to distribute tuples to down stream processors based > on a key such that keys are grouped and can end up at the same destination > JVM. This is handy if you want consistent processing (say updating a > global state store - one thing should do that per key) and horizontal > scalability. In Kafka Streams, each application works in isolation so > multiple instances could be updating state. Is there any example pattern > on Streams that I have missed or is the way to do this in the Kafka World > to distribute by key to topic partitions? > > Thanks, > > Kris >
signature.asc
Description: OpenPGP digital signature
