Hi Guy, Indeed, I referenced the wrong source-code, sorry about that ;-) I created KAFKA-5717 for this.
Thanks Bart On Tue, Aug 8, 2017 at 8:08 PM, Damian Guy <damian....@gmail.com> wrote: > The change logger is not used during restoration of the in-memory-store. > Restoration is handled > https://github.com/apache/kafka/blob/0.11.0/streams/src/ > main/java/org/apache/kafka/streams/state/internals/ > InMemoryKeyValueStore.java#L79 > > But, even then it is just putting `null` when it should be deleting it. > Feel free to raise a JIRA > Thanks, > Damian > > On Tue, 8 Aug 2017 at 12:09 Bart Vercammen <b...@cloutrix.com> wrote: > > > That's RocksDB .. I'm using in-memory stores ... > > here: > > > > https://github.com/apache/kafka/blob/0.11.0/streams/src/ > main/java/org/apache/kafka/streams/state/internals/ > ChangeLoggingKeyValueBytesStore.java#L56 > > the 'null' is not checked ... > > > > On Tue, Aug 8, 2017 at 12:52 PM, Damian Guy <damian....@gmail.com> > wrote: > > > > > Hi, > > > The null values are treated as deletes when they are written to the > > store. > > > You can see here: > > > https://github.com/apache/kafka/blob/0.11.0/streams/src/ > > > main/java/org/apache/kafka/streams/state/internals/ > RocksDBStore.java#L261 > > > > > > On Tue, 8 Aug 2017 at 11:22 Bart Vercammen <b...@cloutrix.com> wrote: > > > > > > > Hi, > > > > > > > > I noticed the following: > > > > When a kafka streams application starts, it will restore its state in > > its > > > > state-stores (from the log-compacted kafka topic). All good so far, > > but > > > I > > > > noticed that the 'deleted' entries are actually read in into the > store > > as > > > > 'key' with value:`null` > > > > > > > > Is this expected behaviour? I would assume that 'null' values are > > > ignored > > > > when restoring the state as this is exactly how the entries are > deleted > > > on > > > > the log-compacted kafka-topic. > > > > > > > > When the compaction has run on the kafka topic, all is fine, but when > > the > > > > segment is not compacted yet, these null values are read in. > > > > > > > > Greets, > > > > Bart > > > > > > > >
