Urko, You can enable changelog topics for your state store. This will enable the application to persist the data to a Kafka topic. Next time when application start, it will first build it's state by using this topic. Are you using Kstreams or the low-level processor API?
Regards, Mangat On Mon, May 17, 2021 at 5:30 PM Urko Lekuona <u...@arima.eu> wrote: > Hello, > > I have a question regarding the use of Kafka/Kafka Streams to store the > state of a stateful application. > > My application is filtering on a stream based on a value from the previous > event of the stream. For example, if the previous car with the same model > was red, this car cannot be red. Previously, I was saving this state > in-memory (a map), where I could query the value of the previous event and > overwrite it if necessary. > > The problem with this implementation is that I lost the state if my > application crashed, which is something I can't afford. I thought about > storing it in a DB, but as I'm using Kafka, I've decided, and managed, to > store it there. Basically, I'm creating a GlobalKTable and using the > Streams facade (KafkaStreams.store) to query it's contents. This is > working, but I'm not familiarized with the Kafka environment enough to > understand it's implications. > > So that's why I come for help. Am I doing this right? Should or shouldn't I > do this? > > P.S. Message retention time shouldn't be an issue in my case, as this > information expires after a day, so as long as I can retain the events that > long, I'll be fine. > > Thanks in advance, > > Urko >
