pnowojski commented on code in PR #20485:
URL: https://github.com/apache/flink/pull/20485#discussion_r964443840
##########
flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/reader/splitreader/SplitReader.java:
##########
@@ -68,4 +69,29 @@
* @throws Exception if closing the split reader failed.
*/
void close() throws Exception;
+
+ /**
+ * Pauses or resumes reading of individual splits readers.
+ *
+ * <p>Note that no other methods can be called in parallel, so it's fine
to non-atomically
+ * update subscriptions. This method is simply providing connectors with
more expressive APIs
+ * the opportunity to update all subscriptions at once.
+ *
+ * <p>This is currently used to align the watermarks of splits, if
watermark alignment is used
+ * and the source reads from more than one split.
+ *
+ * <p>The default implementation throws an {@link
UnsupportedOperationException} where the
+ * default implementation will be removed in future releases. To be
compatible with future
+ * releases, it is recommended to implement this method and override the
default implementation.
+ *
+ * @param splitsToPause the splits to pause
+ * @param splitsToResume the splits to resume
+ */
+ default void pauseOrResumeSplits(
+ Collection<SplitT> splitsToPause, Collection<SplitT>
splitsToResume) {
+ throw new UnsupportedOperationException(
+ "This split reader does not support pause or resume. (To use
watermark alignment "
+ + "and allow unaligned source splits set configuration
parameter "
+ +
"'pipeline.watermark-alignment.allow-unaligned-source-splits' to true.)");
Review Comment:
Sounds great!
> "watermark alignment if there is more than a single split."
change to:
> "watermark alignment if there is more than a single split per split
reader."
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
