Re: Parse checkpoint _metadata file

2022-12-22 Thread Evgeniy Lyutikov
date and size). About 90% of links lead to non-existent objects. От: Hangxiang Yu Отправлено: 22 декабря 2022 г. 14:21:46 Кому: Evgeniy Lyutikov; user@flink.apache.org Тема: Re: Parse checkpoint _metadata file Hi, > Is there some way to deserialize the checkpoin

Re: Parse checkpoint _metadata file

2022-12-21 Thread Hangxiang Yu
Hi, > Is there some way to deserialize the checkpoint _metadata file? You could use some methods like SavepointLoader#loadSavepointMetadata in the State processor api to load it. > If i try to process the file with regular expressions, then approximately 90% of S3 paths of objects are actually mis

Parse checkpoint _metadata file

2022-12-21 Thread Evgeniy Lyutikov
Hello All Is there some way to deserialize the checkpint _metadata file? I want to understand what the checkpoint saves and how the occupied space is distributed. If i try to process the file with regular expressions, then approximately 90% of S3 paths of objects are actually missing in the buc