pnowojski commented on pull request #18790: URL: https://github.com/apache/flink/pull/18790#issuecomment-1046915190
I've chatted offline with @fapaul , and we both reached very similar conclusions to what I've posted above. Generic solution to this problem seems to be also described in the FLIP-27 ( https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface#FLIP27:RefactorSourceInterface-OverallDesign ), with sending the global watermark event from the SourceCoordinator/SplitEnumerator to the SourceOperators. 1. Source readers/operators start with the "global watermark"/"max allowed watermark" set to MIN_VALUE 2. SourceCoordinator can decide when to bump the "global watermark"/"max allowed watermark". In this case, it should bump it to MAX_VALUE after the initial/starting set of splits has been generated and sent to readers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
