Hi, > Is there some way to deserialize the checkpoint _metadata file? You could use some methods like SavepointLoader#loadSavepointMetadata in the State processor api to load it.
> If i try to process the file with regular expressions, then approximately 90% of S3 paths of objects are actually missing in the bucket and I would like to understand how it works and how it is restored if there are links to missing files. I may miss something. How did you find "90% of S3 paths of objects are missing" ? If you have stopped the job, you could find all files related to the checkpoint using the above method. After you list all files in the checkpoint dir and compared with above, you may also get the remaining files list. On Wed, Dec 21, 2022 at 9:56 PM Evgeniy Lyutikov <eblyuti...@avito.ru> wrote: > Hello All > Is there some way to deserialize the checkpint _metadata file? > > I want to understand what the checkpoint saves and how the occupied space > is distributed. > > If i try to process the file with regular expressions, then approximately > 90% of S3 paths of objects are actually missing in the bucket and I would > like to understand how it works and how it is restored if there are > links to missing files. > > We use Flink 1.14.4 > > Thanks > > > > * ------------------------------ *“This message contains confidential > information/commercial secret. If you are not the intended addressee of > this message you may not copy, save, print or forward it to any third party > and you are kindly requested to destroy this message and notify the sender > thereof by email. > Данное сообщение содержит конфиденциальную информацию/информацию, > являющуюся коммерческой тайной. Если Вы не являетесь надлежащим адресатом > данного сообщения, Вы не вправе копировать, сохранять, печатать или > пересылать его каким либо иным лицам. Просьба уничтожить данное сообщение и > уведомить об этом отправителя электронным письмом.” > -- Best, Hangxiang.
