Best,
Dawid On 13/04/2022 11:27, Sebastian Mattheis wrote:
Dear Flink developers, I would like to open a discussion on FLIP 217 [1] for an extension of Watermark Alignment to perform alignment also in SplitReaders. To do so, SplitReaders must be able to suspend and resume reading from split sources where the SourceOperator coordinates and controlls suspend and resume. To gather information about current watermarks of the SplitReaders, we extend the internal WatermarkOutputMulitplexer and report watermarks to the SourceOperator. There is a PoC for this FLIP [2], prototyped by Arvid Heise and revised and reworked by Dawid Wysakowicz (He did most of the work.) and me. The changes are backwards compatible in a way that if affected components do not support split alignment the behavior is as before. Best, Sebastian [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-217+Support+watermark+alignment+of+source+splits [2] https://github.com/dawidwys/flink/tree/aligned-splits
OpenPGP_signature
Description: OpenPGP digital signature
