I will try to clarify what I mean by "old state that is not longer needed". Lets say I consume messages about products that has been sold to customers and I keep a KTable with aggregated data for product with a specific id and the number of times it has been bought. At some point a product that has been bought many times is no longer available and will never be again - let's day it's deleted. Then this aggregated data about it is old and no longer needed. From your answer I understand that whenever a product is deleted a message by the Kafka Streams application needs to be consumed and that specific entry for that product needs to be deleted from the KTable with aggregated data using tombstone. If I don't do that the entry will never be deleted and will stay in the KTable. Is this correct?
Thanks, Mihaela Stoycheva On Thu, Apr 19, 2018 at 3:12 PM, Matthias J. Sax <matth...@confluent.io> wrote: > Not sure what you mean by "old state that is not longer needed" ? > > key-value entries are kept forever, and there is no TTL. If you want to > delete something from the store, you can return `null` as aggregation > result though. > > -Matthias > > On 4/19/18 2:28 PM, adrien ruffie wrote: > > Hi Mihaela, > > > > > > by default a KTable already have a log compacted behavior. > > > > therefore you don't need to manually clean up. > > > > > > Best regards, > > > > > > Adrien > > > > ________________________________ > > De : Mihaela Stoycheva <mihaela.stoych...@gmail.com> > > Envoyé : jeudi 19 avril 2018 13:41:22 > > À : users@kafka.apache.org > > Objet : Is KTable cleaned up automatically in a Kafka streams > application? > > > > Hello, > > > > I have a Kafka Streams application that is consuming from two topics and > > internally aggregating, transforming and joining data. I am using KTable > as > > result of aggregation and my question is if KTables are cleaned using > some > > mechanism of Kafka Streams or is this something that I have to do > manually > > - clean up old state that is not longer needed? > > > > Regards, > > Mihaela Stoycheva > > > >
