Dawid Wysakowicz created FLINK-37025: ----------------------------------------
Summary: Periodic SQL watermarks can travel back in time Key: FLINK-37025 URL: https://issues.apache.org/jira/browse/FLINK-37025 Project: Flink Issue Type: Bug Components: Table SQL / Runtime Affects Versions: 2.0-preview, 1.20.0 Reporter: Dawid Wysakowicz Fix For: 2.0.0, 1.20.1 If watermarks are generated through SQL, e.g. using `WATERMARK FOR ts AS ts` is used https://github.com/apache/flink/blob/e9353319ad625baa5b2c20fa709ab5b23f83c0f4/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/generated/GeneratedWatermarkGeneratorSupplier.java#L86 The code in {{onEvent}} can move the {{currentWatermark}} back in time, because it does not check if it advanced. This is fine in case of `on-event` watermarks`, because underlying {{WatermarkOutput}} will, but it has confusing outcomes for `on-periodic`. Example: If events with timestamps: ``` {"id":1,"ts":"2024-01-01 00:00:00"} {"id":3,"ts":"2024-01-03 00:00:00"} {"id":2,"ts":"2024-01-02 00:00:00"} ``` come within the periodic emit interval, the generated watermark will be "2024-01-02 00:00:00" instead of "2024-01-03 00:00:00". As a result the watermark for "2024-01-03 00:00:00" is swallowed and a new event is required to progress the processing of that event. -- This message was sent by Atlassian Jira (v8.20.10#820010)