Hi Roman,
Very thanks for the feedbacks and suggestions!
> I think UC will be the common case with multiple sources each with
DoP > 1.
> IIUC, waiting for EoP will be needed on each subtask each time one of
it's source subtask finishes.
Yes, waiting for EoP would be required for each input channel if we do
not blocking the upstream
finished task specially.
> Yes, but checkpoint completion notification will not be sent until all
the EOPs are processed.
The downstream tasked get triggered indeed must wait for received EoPs
from all the input channels,
I initially compared it with the completely aligned cases and now the remaining
execution graph after the
trigger task could still taking normal unaligned checkpoint (like if A -> B ->
C -> D, A get finished and B get
triggered, then B -> C -> D could still taking normal unaligned checkpoint).
But still it could not limit the
possible max delay.
> Not all declines cause job failure, particularly
CHECKPOINT_DECLINED_TASK_NOT_READY doesn't.
Sorry for mistaken the logic here and CHECKPOINT_DECLINED_TASK_NOT_READY
indeed do not cause failure.
But since after a failed checkpoint we would have to wait for the checkpoint
interval for the next checkpoint, I also
agree the following option would be a better one that we try to complete each
checkpoint.
>> Thus another possible option might be let the upstream task to wait till all
>> the pending buffers in the result partition has been flushed before get to
>> finish.
> This is what I meant by "postpone JM notification from source". Just blocking
> the task thread wouldn't add much complexity, though I'm not sure if it would
> cause any problems.
>> do you think it would be ok for us to view it as an optimization and
>> postpone it to future versions ?
> I think that's a good idea.
And also very sorry for here I should wrongly understand the proposals, and
currently
I also do not see explicit problems for waiting for the flush of pipeline
result partition.
Glad that we have the same viewpoints on this issue. :)
Best,
Yun
------------------------------------------------------------------
From:Khachatryan Roman <khachatryan.ro...@gmail.com>
Send Time:2021 Jan. 11 (Mon.) 19:14
To:Yun Gao <yungao...@aliyun.com>
Cc:dev <d...@flink.apache.org>; user <user@flink.apache.org>
Subject:Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished
Hi Yun,
> b) With unaligned checkpoint enabled, the slower cases might happen if the
> downstream task processes very slowly.
I think UC will be the common case with multiple sources each with DoP > 1.
IIUC, waiting for EoP will be needed on each subtask each time one of it's
source subtask finishes.
> But since only the result partition part of the finished upstream need wait
> to be processed, the other part of
> the execution graph could still perform the unaligned checkpoint normally
Yes, but checkpoint completion notification will not be sent until all the EOPs
are processed.
> Declining the RPC-trigger checkpoint would indeed simplify the
> implementation, but since currently by default the
> failed checkpoint would cause job failover, thus we might have some concerns
> in directly decline the checkpoint.
Not all declines cause job failure, particularly
CHECKPOINT_DECLINED_TASK_NOT_READY doesn't.
> Thus another possible option might be let the upstream task to wait till all
> the pending buffers in the result partition has been flushed before get to
> finish.
This is what I meant by "postpone JM notification from source". Just blocking
the task thread wouldn't add much complexity, though I'm not sure if it would
cause any problems.
> do you think it would be ok for us to view it as an optimization and postpone
> it to future versions ?
I think that's a good idea.
Regards,
Roman
