1996fanrui commented on code in PR #27182:
URL: https://github.com/apache/flink/pull/27182#discussion_r2534711616
##########
flink-tests/src/test/java/org/apache/flink/test/state/ChangelogRecoveryCachingITCase.java:
##########
@@ -95,6 +96,9 @@ public void before() throws Exception {
Configuration configuration = new Configuration();
configuration.set(CACHE_IDLE_TIMEOUT, Duration.ofDays(365)); // cache
forever
+ // Disable UNALIGNED_ALLOW_ON_RECOVERY since the output buffer states
file may be opened
+ // from multiple subtasks
+ configuration.set(UNALIGNED_ALLOW_ON_RECOVERY, false);
Review Comment:
IIUC, it has been included in the 5th commit:
https://github.com/apache/flink/pull/27182/commits/130e81bfd94fe0b51a3731536ce2e9c548cfc646
##########
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/channel/RecoveredChannelStateHandler.java:
##########
@@ -99,7 +100,12 @@ public void recover(
Buffer buffer = bufferWithContext.context;
try {
if (buffer.readableBytes() > 0) {
- for (final RecoveredInputChannel channel :
getMappedChannels(channelInfo)) {
+ List<RecoveredInputChannel> mappedChannels =
getMappedChannels(channelInfo);
+ checkState(
+ mappedChannels.size() == 1,
+ "One buffer is only distributed to one target
InputChannel since "
+ + "one buffer is expected to be processed once
by the same task.");
+ for (final RecoveredInputChannel channel : mappedChannels) {
Review Comment:
It is an existing limitation.
Before this PR, the `InputChannelRecoveredStateHandler` is responsible for
distribute one input buffer to input channels of same task. From an
implementation perspective, the same input buffer belongs to the same Virtual
Channel. If it is distributed to multiple input channels of a task, it will be
consumed repeatedly.
After this PR, the upstream output buffers are also distributed by
`InputChannelRecoveredStateHandler`, I would limit it explicitly to avoid
potential bugs.
> Shouldn't we use sth like Iterables.getOnlyElement?
Good point, updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
