Re: Issues with watermark alignment in Flink 1.15

2022-07-14 Thread Jun Qin
Found the reason: it does not work together with .withIdleness(Duration.ofSeconds(1)) Isn't this a valid scenario: one subtask has multiple input streams/channels where some are idle, others have large watermark skews? In addition, do we expect that the watermark update interval in: .withWaterm

Issues with watermark alignment in Flink 1.15

2022-07-10 Thread Jun Qin
Hi All I am trying watermark alignment in Flink 1.15 with: watermarkStrategy = WatermarkStrategy.<~>forBoundedOutOfOrderness( Duration.ofMillis(outOfOrderness)) .withWatermarkAlignment("wm-group", Duration.ofSeconds(10), Duration.ofSeconds(1)) .withTimestampAssign