Re: In-flight data within aligned checkpoints/savepoints

2022-08-23 Thread Darin Amos via user
I think you are right that I am just seeing "processed in-flight data" and I was misreading the UI. However, that raises another concern I have. When I deploy a new build that contains changes to the classes for the in-flight data, my deployments have failed when I restore from checkpoints (or sav

Re: In-flight data within aligned checkpoints/savepoints

2022-08-22 Thread yanfei lei
Hi Darin, > I often see my checkpoints contain "Processed (persisted) in-flight data". The values outside the parentheses represent Processed in-flight data[1], and the values inside the parentheses represent persisted in-flight data[1] , what kind of case did you see in your WEB UI?If the >0 val

In-flight data within aligned checkpoints/savepoints

2022-08-22 Thread Darin Amos via user
Hi All! I am running Flink 1.13.1 and I have unaligned checkpoints disabled, however I often see my checkpoints contain "Processed (persisted) in-flight data". According to the Flink documentation, it states: - *Persisted in-flight data: The number of bytes persisted during the alignment (t