I'm also having the opposite problem -- my unit test moves the watermark to just before a timestamped input. I was expecting the DoFn to execute for all windows up to that watermark, seeing nothing in the side input.
However, that isn't the case -- the test shows that the DoFn side input *before* the watermark, contains the input timestamped *after* the watermark. This makes no sense to me. Regards, Raman On Sat, Apr 10, 2021 at 2:46 AM Raman Gupta <rocketra...@gmail.com> wrote: > I have a topology that looks like this: > > input -> 1hr window -> view > | > -> 5m window -> dofn(view) > > IOW -- an input is windowed into both 1 hour windows, as well as 5 minute > windows. The 1 hour window data is collected into a view. > > Meanwhile, the input is also windowed with 5 minute windows, there is some > processing, and then finally a DoFn that uses the view as a side input. > > Both windows allow late data, and fire immediately on a late element. My > expectation would have been that any processing occurring in the DoFn after > the processing time of the late element would see the late element in the > view. However, that doesn't seem to be the case. Why? > > Is there another approach I can use that doesn't suffer from this problem? > > Regards, > Raman Gupta > >
