rkhachatryan commented on code in PR #25167:
URL: https://github.com/apache/flink/pull/25167#discussion_r1715522711
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/source/ProgressiveTimestampsAndWatermarks.java:
##########
@@ -273,6 +322,16 @@ void emitPeriodicWatermark() {
}
watermarkMultiplexer.onPeriodicEmit();
}
+
+ public void pauseOrResumeSplits(
+ Collection<String> splitsToPause, Collection<String>
splitsToResume) {
+ for (String splitId : splitsToPause) {
+ inputActivityClocks.get(splitId).markBlocked();
+ }
+ for (String splitId : splitsToResume) {
+ inputActivityClocks.get(splitId).markUnblocked();
+ }
Review Comment:
I'm not talking about synchronizing threads accessing `SourceOperator`
fields.
Let me rephrase my concern:
1. Output for split is created, split is added to `inputActivityClocks` and
`splitCurrentWatermarks` maps
2. data is fetched
3. `SourceReaderBase.moveToNextSplit()` sees no data at some point
-> call `ProgressiveTimestampsAndWatermarks.releaseOutputForSplit()`
-> remove the split from `inputActivityClocks`(source coordinator is not
aware of this)
5. `SourceCoordinator` sends `WatermarkAlignmentEvent`
-> call `SourceOperator.checkSplitWatermarkAlignment()` on TM
-> iterate over `splitCurrentWatermarks` and
-> calls `ProgressiveTimestampsAndWatermarks.pauseOrResumeSplits()`
-> NPE because the split was removed from `inputActivityClocks` map
Isn't this scenario possible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
