Re: does Kafka exactly-once guarantee also apply to kafka state stores?

2021-03-24 Thread Matthias J. Sax
Well, as the ticket suggests, you cannot use a remote store with EOS. There is no workaround. We would need to address the ticket you linked to allow using remote stores. Btw: this ticket might actually be more relevant for supporting remove stores: https://issues.apache.org/jira/browse/KAFKA-1254

Re: Kafka Streams Processor API state stores not restored via changelog topics

2021-03-24 Thread Upesh Desai
Hi Guozhang, Here are some of the answers to your questions I see during my testing: 1. ChangelogMetadata#restoreEndOffset == 1 ; This is expected as in my test 1 record had been added to the store. However the numRecords variable is still set to 0 2. For that particular test, `hasRestor

Re: Kafka Streams Processor API state stores not restored via changelog topics

2021-03-24 Thread Guozhang Wang
Hello Upesh, Thanks for the detailed report. I looked through the code and tried to reproduce the issue, but so far have not succeeded. I think I may need some further information from you to help my further investigation. 1) The `bufferedLimitIndex == 0` itself does not necessarily mean there's