rkhachatryan commented on a change in pull request #11507: [FLINK-16587] Add
basic CheckpointBarrierHandler for unaligned checkpoint
URL: https://github.com/apache/flink/pull/11507#discussion_r408144900
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/CheckpointedInputGate.java
##########
@@ -171,6 +176,18 @@ else if (bufferOrEvent.getEvent().getClass() ==
CancelCheckpointMarker.class) {
}
}
+ public List<Buffer> requestInflightBuffers(long checkpointId, int
channelIndex) throws IOException {
+ if (((CheckpointBarrierUnaligner)
barrierHandler).hasInflightData(checkpointId, channelIndex)) {
+ return
inputGate.getChannel(channelIndex).requestInflightBuffers(checkpointId);
Review comment:
I mean the case when:
1. task thread is suspended by OS scheduler
1. netty thread receives a buffer `X` and adds it to
`RemoteInputChannel.receivedBuffers`
1. netty thread receives a barrier and schedules a checkpoint by enqueueing
a mail (`RIC.onBuffer` -> `ThreadSafeUnaligner#notifyBarrierReceived`)
1. task thread wakes up and sees a **mail** (not a default action);
1. task thread performs a checkpoint; it checks
`CheckpointBarrierUnaligner.hasInflightData` and reads `false` so it doesn't go
into `requestInflightBuffers`
Now:
- The buffer `X` added in the beginning is missing in the checkpoint;
- `RemoteInputChannel#lastRequestedCheckpointId` is not updated;
Am I missing something?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
