Hi, Your example does not show what watermarks are flowing through the program. It prints the watermark at the point a record is being emitted. As the cited text states, the final watermark is emitted after all records are emitted. You can test it e.g. with the newly added writeWatermark method in 1.14 or by implementing a ProcessFunction with a timer for Long.MAX_VALUE, or lastly with a custom operator.
Best, Dawid On 08/07/2021 14:51, Yik San Chan wrote: > Hi, > > According to the docs [1] > > When a source reaches the end of the input, it emits a final watermark > with timestamp Long.MAX_VALUE, indicating the "end of time". > > > However, in my small experiment [2], the Flink job reads from a local > csv file, and prints a watermark for each record in the SinkFunction > `invoke` method. Even though I expect the last record comes with a > MAX_VALUE watermark, all records actually come with a MIN_VALUE watermark. > > > ``` > > watermark=-9223372036854775808 > watermark=-9223372036854775808 > 1 > 10 > > ``` > > > I wonder what I miss? Is there a way to make sure the source generates > a MAX_VALUE watermark after it finishes all records? > > Thank you! > > [1] > https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-core/src/main/java/org/apache/flink/api/common/eventtime/Watermark.java#L39 > <https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-core/src/main/java/org/apache/flink/api/common/eventtime/Watermark.java#L39> > > [2] https://github.com/YikSanChan/flink-batch-source-watermark > <https://github.com/YikSanChan/flink-batch-source-watermark>
OpenPGP_signature
Description: OpenPGP digital signature
