The nulls are expected. It's not about expired session windows though: sessions window are stored as `<(key,start-timestamp,end-timestamp), value>`. If the window boundaries changed due to new incoming events (or maybe a merge of two windows due to late arriving records), the window is updated via a delete of the old window and an create of the new window (because it's two different keys for old and new window).
Thus, using session window does not seem to be the best approach for your problem. Check out this example on how to do event de-duplication: https://github.com/confluentinc/kafka-streams-examples/blob/5.0.1-post/src/test/java/io/confluent/examples/streams/EventDeduplicationLambdaIntegrationTest.java Hope this helps. -Matthias On 12/2/18 4:59 PM, Liam Clarke wrote: > Hi all, > > Using > > I am using a session window of 1 minute to detect and prevent duplicated > events within the window using reduce() to preserve the first seen value > and accumulate how often duplicates were seen, I'm then converting the > windowed KTable into a stream to continue processing events. > > However, later in the stream, I've been hitting NPEs because after the > window expiry, the stream is emitting a previously seen key with a null > value when I'd assumed any values in the stream would be non-null. Is this > an idiom expressing that the window for that key has expired and as such > needs to be explicitly handled? > > I'm worried that my filtering those values from the ongoing stream is > hiding a problem further upstream, but if it's working as designed, then > sweet as. > > Kind regards, > > Liam Clarke >
signature.asc
Description: OpenPGP digital signature
