Re: flight buffer local storage

[Enric Ott]

Thanks,Zhanghao.
I think it's the async upload mechanism helped mitigating the in flight buffers 
materialization latency,and the execution vertex restarting procedure just 
reads the in flight buffers and the local  TaskStateSnapshots to make its 
job done.




------------------ Original ------------------
From:                                                                           
                                             "Zhanghao Chen"                    
                                                                
<zhanghao.c...@outlook.com&gt;;
Date:&nbsp;Sun, Jul 21, 2024 03:56 PM
To:&nbsp;"Enric 
Ott"<243816...@qq.com&gt;;"user@flink.apache.org"<user@flink.apache.org&gt;;

Subject:&nbsp;Re: flight buffer local storage



  By default, Flink uses aligned checkpoint where we wait for all in-flight 
data before the barriers to be fully processed and then make the checkpoints. 
There's no in need to store in-flight buffers in this case at the cost of 
additional barrier alignment, which  may take a long time at the presence of 
backpressure. Unaligned checkpoint is introduced to solve this problem, where 
in-flight buffers are stored in cp without the need of alignment.
  
 
   Best,
 Zhanghao Chen
 
 
 
 
 From: Enric Ott <243816...@qq.com&gt;
 Sent: Wednesday, July 17, 2024 16:35
 To: user@flink.apache.org <user@flink.apache.org&gt;
 Subject: flight buffer local storage &nbsp;
 
 Hello,Community: &nbsp; Why doesn't flink store in flight buffers to local 
disks when it checkpoints?
 &nbsp; Thanks.