Hi Leonard, Let's assume we have two streams. S1 - id, value1, ts1 with watermark = ts1 - 1 S2 - id, value2, ts2 with watermark = ts2 - 1
Then we have following interval join SELECT id, value1, value2, ts1 FROM S1 JOIN S2 ON S1.id = S2.id and ts1 between ts2 - 1 and ts2 Let's have events. stream, id, value, ts S1, id1, v1, 1 S2, id1, v2, 1 For this events and internal join Flink will emit an event in the output stream: id1, v1, v2, 1 Despite the fact the watermark for both streams is not reached. Now similar situation for Event Time Temporal Join SELECT id, value1, value2, ts1 FROM S1 JOIN S2 FOR SYSTEM_TIME AS OF S1.ts1 ON S1.id = S2.id Let's have events. S1, id1, v1, 1 S2, id1, v2, 1 Nothing is happening as none of the streams have reached the watermark. Now let's add S2, id2, v2, 101 This should trigger join for id1 because we have all the knowledge to perform this join (we know that the watermark for id1 record was reached). Unfortunately to trigger join on id1 we also need a watermark on S1 side and I think this behaviour is wrong. I hope I explained everything correctly. Regards, Maciek wt., 27 kwi 2021 o 08:58 Leonard Xu <xbjt...@gmail.com> napisał(a): > > Hello, Maciej > > I agree the watermark should pass on versioned table side, because > this is the only way to know which version of record should be used. > But if we mimics behaviour of interval join then main stream watermark > could be skipped. > > > IIRC, rowtime interval join requires the watermark on both sides, and the > watermark > will be used to clean up the outdated data and advance the data progress both > in rowtime interval join and rowtime temporal join. > > Best, > Leonard > -- Maciek Bryński
