rkhachatryan commented on a change in pull request #18324:
URL: https://github.com/apache/flink/pull/18324#discussion_r790718434
##########
File path:
flink-dstl/flink-dstl-dfs/src/main/java/org/apache/flink/changelog/fs/FsStateChangelogWriter.java
##########
@@ -302,14 +302,16 @@ public void reset(SequenceNumber from, SequenceNumber to)
{
}
private static ChangelogStateHandleStreamImpl buildHandle(
- KeyGroupRange keyGroupRange, NavigableMap<SequenceNumber,
UploadResult> results) {
+ KeyGroupRange keyGroupRange,
+ NavigableMap<SequenceNumber, UploadResult> results,
+ long incrementalSize) {
List<Tuple2<StreamStateHandle, Long>> tuples = new ArrayList<>();
long size = 0;
for (UploadResult uploadResult : results.values()) {
tuples.add(Tuple2.of(uploadResult.getStreamStateHandle(),
uploadResult.getOffset()));
size += uploadResult.getSize();
}
- return new ChangelogStateHandleStreamImpl(tuples, keyGroupRange, size);
+ return new ChangelogStateHandleStreamImpl(tuples, keyGroupRange, size,
incrementalSize);
Review comment:
I think you're right in the sense that pre-emptively uploaded changes do
not affect sync/async phase.
But I'd say those changes **are** uploaded during that checkpoint.
I think confusion comes from the definition of "during that checkpoint". It
can mean two things:
1. the time from triggering a checkpoint to its completion (i.e. sync+async
phase in task context)
2. the time between two barriers
I think we should explicitly clarify that we mean 1st one the docs and UI:
"during that checkpoint" -> "during sync and async phases of that
checkpoint".
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
