AHeise commented on a change in pull request #11507: [FLINK-16587] Add basic
CheckpointBarrierHandler for unaligned checkpoint
URL: https://github.com/apache/flink/pull/11507#discussion_r399291992
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/StreamTaskNetworkInput.java
##########
@@ -197,6 +205,29 @@ public int getInputIndex() {
return checkpointedInputGate.getAvailableFuture();
}
+ @Override
+ public CompletableFuture<?> prepareSnapshot(long checkpointId) throws
IOException {
+ // Note that if considering recovery in future, we should
guarantee that the spilled buffers in one channel
+ // should be close together because one record might span
multiple buffers.
+ for (int channelIndex = 0; channelIndex <
recordDeserializers.length; channelIndex++) {
+ final InputChannel channel =
checkpointedInputGate.getChannel(channelIndex);
+
+
recordDeserializers[channelIndex].getUnconsumedBuffer().ifPresent(buffer ->
+ channelStateWriter.addInputData(
+ checkpointId,
+ channel.getChannelInfo(),
+
ChannelStateWriter.SEQUENCE_NUMBER_UNKNOWN,
+ buffer));
+
+ channelStateWriter.addInputData(
+ checkpointId,
+ channel.getChannelInfo(),
+ ChannelStateWriter.SEQUENCE_NUMBER_UNKNOWN,
+
checkpointedInputGate.getInflightBuffers(checkpointId,
channelIndex).toArray(new Buffer[0]));
+ }
+ return
checkpointedInputGate.getAllBarriersReceivedFuture(checkpointId);
+ }
+
@Override
public void close() throws IOException {
Review comment:
Interesting idea. However, would trigger `complete` not more harm than it
would solve? When we trigger `complete`, it might create an incomplete
checkpoint...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
