This might help: https://kafka.apache.org/documentation/#compaction
On Wed, 26 Jul 2017 at 12:37 Sameer Kumar <sam.kum.w...@gmail.com> wrote: > Damian, > > Does this mean data is retained for infinite time limited only by disk > space. > > -Sameer. > > On Wed, Jul 26, 2017 at 3:53 PM, Sameer Kumar <sam.kum.w...@gmail.com> > wrote: > > > got it. Thanks. > > > > On Wed, Jul 26, 2017 at 3:24 PM, Damian Guy <damian....@gmail.com> > wrote: > > > >> The changelog is one created by kafka streams, then it is a compacted > >> topic > >> and the retention period is irrelevant. If it is one you have created > >> yourself and isn't compacted, then the data will be retained in the > topic > >> for as long as the retention period. > >> If you use a non-compacted topic and the kafka-streams instance crashes > >> then that data may be lost from the state store as it will use the topic > >> to > >> restore its state. > >> > >> On Wed, 26 Jul 2017 at 10:24 Sameer Kumar <sam.kum.w...@gmail.com> > wrote: > >> > >> > ok. Thanks. > >> > > >> > Actually, I had this confusion. Changelog like every Kafka topic would > >> have > >> > its retention period, lets say 2 days. and if the value on day1 for > >> key1 = > >> > 4 and data for key1 doesnt come for next 3 days. Would it still retail > >> the > >> > same value(key1=4) on day4. > >> > > >> > -Sameer. > >> > > >> > On Wed, Jul 26, 2017 at 2:22 PM, Damian Guy <damian....@gmail.com> > >> wrote: > >> > > >> > > Sameer, > >> > > > >> > > For a KeyValue store the changelog topic is a compacted topic so > >> there is > >> > > no retention period. You will always retain the latest value for a > >> key. > >> > > > >> > > Thanks, > >> > > Damian > >> > > > >> > > On Wed, 26 Jul 2017 at 08:36 Sameer Kumar <sam.kum.w...@gmail.com> > >> > wrote: > >> > > > >> > > > Hi, > >> > > > > >> > > > Retention period for state stores are clear(default, otherwise > >> > specified > >> > > by > >> > > > TimeWindows.until). Intrigued to know the retention period for key > >> > > values. > >> > > > > >> > > > The use case is something like I am reading from a windowed store, > >> and > >> > > > using plain reduce() with out any time windows. Would the values > be > >> > > > retained foreever. > >> > > > > >> > > > -Sameer. > >> > > > > >> > > > >> > > >> > > > > >
