A. Sophie Blee-Goldman created KAFKA-12475:
----------------------------------------------

             Summary: Kafka Streams breaks EOS with remote state stores
                 Key: KAFKA-12475
                 URL: https://issues.apache.org/jira/browse/KAFKA-12475
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: A. Sophie Blee-Goldman


Currently in Kafka Streams, exactly-once semantics (EOS) require that the state 
stores be completely erased and restored from the changelog from scratch in 
case of an error. This erasure is implemented by closing the state store and 
then simply wiping out the local state directory. This works fine for the two 
store implementations provided OOTB, in-memory and rocksdb, but fails when the 
application includes a custom StateStore based on remote storage, such as 
Redis. In this case Streams will fail to erase any of the data before 
reinserting data from the changelog, resulting in possible duplicates and 
breaking the guarantee of EOS.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to