mxm commented on code in PR #20485:
URL: https://github.com/apache/flink/pull/20485#discussion_r963759026
##########
flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/reader/splitreader/SplitReader.java:
##########
@@ -68,4 +69,29 @@
* @throws Exception if closing the split reader failed.
*/
void close() throws Exception;
+
+ /**
+ * Pauses or resumes reading of individual splits readers.
+ *
+ * <p>Note that no other methods can be called in parallel, so it's fine
to non-atomically
+ * update subscriptions. This method is simply providing connectors with
more expressive APIs
+ * the opportunity to update all subscriptions at once.
+ *
+ * <p>This is currently used to align the watermarks of splits, if
watermark alignment is used
+ * and the source reads from more than one split.
+ *
+ * <p>The default implementation throws an {@link
UnsupportedOperationException} where the
+ * default implementation will be removed in future releases. To be
compatible with future
+ * releases, it is recommended to implement this method and override the
default implementation.
+ *
+ * @param splitsToPause the splits to pause
+ * @param splitsToResume the splits to resume
+ */
+ default void pauseOrResumeSplits(
+ Collection<SplitT> splitsToPause, Collection<SplitT>
splitsToResume) {
+ throw new UnsupportedOperationException(
+ "This split reader does not support pause or resume. (To use
watermark alignment "
+ + "and allow unaligned source splits set configuration
parameter "
+ +
"'pipeline.watermark-alignment.allow-unaligned-source-splits' to true.)");
Review Comment:
Thanks, have modified it slightly:
```java
throw new UnsupportedOperationException(
"This split reader does not support pausing or resuming
splits which can lead to unaligned splits.\n"
+ "Unaligned splits are splits where the output
watermarks of the splits have diverged more than the allowed limit.\n"
+ "It is highly discouraged to use unaligned source
splits, as this leads to unpredictable\n"
+ "watermark alignment if there is more than a
single split. It is recommended to implement pausing splits\n"
+ "for this source. At your own risk, you can allow
unaligned source splits by setting the\n"
+ "configuration parameter
`pipeline.watermark-alignment.allow-unaligned-source-splits' to true.\n"
+ "Beware that this configuration parameter will be
dropped in a future Flink release.");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
