In case we opt to choose some third party store instead of kafka's stores for storing state (e.g. Redis cache or Ignite), then will we lose the exactly-once guarantee provided by kafka and the state stores can be in an inconsistent state ?
On Sat, Jan 2, 2021 at 4:56 AM Ning Zhang <ning2008w...@gmail.com> wrote: > The physical store behind "state store" is change-log kafka topic. In > Kafka stream, if something fails in the middle, the "state store" is > restored back to the state before the event happens at the first step / > beginning of the stream. > > > > On 2020/12/31 08:48:16, Pushkar Deole <pdeole2...@gmail.com> wrote: > > Hi All, > > > > We use Kafka streams and may need to use exactly-once configuration for > > some of the use cases. Currently, the application uses either local or > > global state store to store state. > > So, the application will consume events from source kafka topic, process > > the events, for state stores it will use either local or global state > store > > of kafka, then produce events onto the destination topic. > > > > Question i have is: in the case of exactly-once setting, kafka streams > > guarantees that all actions happen or nothing happens. So, in this case, > > any state stored on the local or global state store will also be counted > > under 'all or nothing' guarantee e.g. if event is consumed and state > store > > is updated, however some issue occurs before event is produced on > > destination topic then will state store be restored back to the state > > before it was updated for this event? > > >
