Re: Need help with finding inner workings of watermark stream idleness

2022-02-02 Thread Jeff Carter
Thanks, Seth. Yea this looks perfect. I had a feeling I'd need to get deep into things, and no time like the present haha. May ask for more guidance with those inner workings to get a bit of a road map. But that gets into the feature idea and beyond the scope of this thread's original question so

Re: Need help with finding inner workings of watermark stream idleness

2022-02-01 Thread Seth Wiesman
Hi Jeff, I think the class you're looking for is StatusWatermarkValve. Note that this is fairly deep into the runtime stack. Seth On Tue, Feb 1, 2022 at 2:34 PM Jeff Carter wrote: > Thanks, Till. > > That definitely helps a bit. I'm still not seeing where there is some idle > variable that the

Re: Need help with finding inner workings of watermark stream idleness

2022-02-01 Thread Jeff Carter
Thanks, Till. That definitely helps a bit. I'm still not seeing where there is some idle variable that the output.markIdle is setting to true (or whatever it sets). Like the ideal thing would be if there is just some "output.isIdle()" that could be called to know if the stream is or isnt idle. Sin

Re: Need help with finding inner workings of watermark stream idleness

2022-01-11 Thread Till Rohrmann
Hi Jeff, I think this happens in the WatermarksWithIdleness [1]. [1] https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/eventtime/WatermarksWithIdleness.java#L73 Cheers, Till On Tue, Jan 11, 2022 at 6:05 PM Jeff Carter wrote: > I'm looking into ma