[
https://issues.apache.org/jira/browse/FLINK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743036#comment-16743036
]
Kostas Kloudas commented on FLINK-11337:
----------------------------------------
Hi [~edRojas], this does not seem to be a problem of the sink. In fact the
sink, simply propagates the watermark that it receives from the
{{SinkFunction.Contex}}. Could you change the jira title and the description to
reflect the root of the problem?
> Incorrect watermark in StreaminFileSink BucketAssigner.Context when used in
> connected stream
> --------------------------------------------------------------------------------------------
>
> Key: FLINK-11337
> URL: https://issues.apache.org/jira/browse/FLINK-11337
> Project: Flink
> Issue Type: Bug
> Components: filesystem-connector
> Affects Versions: 1.7.0
> Reporter: Edward Rojas
> Priority: Major
>
> When StreamingFileSink is used as sink of a connected stream the "invoke"
> method of the sink could be called before the "combinedWatermark" is updated
> with the timestamp of the element currently being processed, resulting on the
> context containing the incorrect watermark value (the Long.MIN_VALUE when
> using "AssignerWithPeriodicWatermarks" for the firsts events in the stream).
> I reproduce this when using a broadcast stream connected to a data stream.
> The broadcast stream is using a custom timestamp extractor that always return
> the Watermark.MAX_VALUE as it's done in a trining example here:
> [https://github.com/dataArtisans/flink-training-exercises/blob/master/src/main/java/com/dataartisans/flinktraining/solutions/datastream_java/broadcast/OngoingRidesSolution.java#L143.]
> This is problematic as the watermark could not be used reliably to compute
> the bucket id based on event time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
