Hi Ian, I suspect it has something to do with your specified topology, in which it triggers the join first, then the aggregation updates.
For example, take a look at this ticket: https://issues.apache.org/jira/browse/KAFKA-4601 As from its printed topology, due to the repartition topic the join operator may be triggered before the aggregation operator, and hence it would cause the Counts table to return `null` indicating it has not received a record yet. Guozhang On Thu, Aug 24, 2017 at 3:17 AM, Ian Duffy <i...@ianduffy.ie> wrote: > Hi All, > > I'm building a streams applications where I wish to take action on the > input when a certain frequency of the input has been seen. > > At the moment the application roughly goes: > > frequency table = Input Stream -> groupByKey -> Count > > input stream with counts = leftJoin frequency table and input stream > > branch input stream with counts where count > some threshold send to some > other topic, if <= some threshold send to delay processing topic. > > delay stream with counts = leftJoin frequency table and delay stream > > branch delay stream with counts where count > some threshold send to some > other topic, if <= some threshold send to delay processing topic. > > I've seen intermittent failures where the frequency table is giving a null > on join, no matter how long I wait content coming back in on the delayed > stream keeps getting nulls for the count when looking up the frequency > table. > > Any ideas why this might be occurring? > > Thanks, > > Ian. > -- -- Guozhang
