Re: Partial updates in Kafka Streams stores

2018-11-30 Thread Matthias J. Sax
Yes, that could happen. If you write into a store, the write _might_ go directly into the changelog topic, too, and for this case, the instance that takes over would see this write and add it to it's store during recovery. Idempotent updates would solve this issue. -Matthias On 11/30/18 12:17 PM

Re: Partial updates in Kafka Streams stores

2018-11-30 Thread Yoshimo
Hello Matthias, thank you for the answer. What if the operations on the stores are idempotent? I am only worried for the scenario where after doing a put in the first store and before updating the second one, the application crashes or there is a network outage. Will the node that picks up the

Re: Partial updates in Kafka Streams stores

2018-11-30 Thread Matthias J. Sax
You can enable exactly-once processing guarantees to guard against inconsistent stores in fail over scenario. -Matthias On 11/30/18 12:57 AM, Yoshimo wrote: > Hello Kafka users, > > I am currently building a Kafka Streams application and I am using a > transform step with two KeyValue stores,