Re: Maximal watermark when two streams are connected

2019-08-22 Thread Fabian Hueske
Hi Sung, There is no switch to configure the WM to be the max of both streams and it would also in fact violate the core principles of the mechanism. Watermarks are used to track the progress of event time in streams. The implementations of operators rely on the fact that (almost) all records tha

Re: Maximal watermark when two streams are connected

2019-08-22 Thread Sung Gon Yi
I use assignerTimestampsAndWatermarks after connecting two streams and it works well. Thank you. > On 22 Aug 2019, at 3:26 PM, Jark Wu wrote: > > Hi Sung, > > Watermark will be advanced only when records come in if you are using > ".assignTimestampsAndWatermarks()". > One way to solve this

Re: Maximal watermark when two streams are connected

2019-08-21 Thread Jark Wu
Hi Sung, Watermark will be advanced only when records come in if you are using ".assignTimestampsAndWatermarks()". One way to solve this problem is you should call ".assignTimestampsAndWatermarks()" before the condition to make sure there are messages. Best, Jark On Thu, 22 Aug 2019 at 13:52, Su

Maximal watermark when two streams are connected

2019-08-21 Thread Sung Gon Yi
Hello, Originally, watermark of connected stream is set by minimum of watermarks two streams when two streams are connected. I wrote a code to connect two streams but one of streams does not have any message by a condition. In this situation, watermark is never increased and processing is stuck.