Hi Yu, are you using incremental checkpoints [1]? If so, then the smaller checkpoints would be the deltas and the larger the complete state.
[1] https://flink.apache.org/features/2018/01/30/incremental-checkpointing.html On Wed, Mar 4, 2020 at 6:41 PM Yu Yang <yuyan...@gmail.com> wrote: > Hi all, > > We have a flink job that does check-pointing per 10 minutes. We noticed > that for the check-points of this job, the _metadata file size can vary a > lot. In some checkpoint, we observe that _metadata file size was >900MB, > while in some other check-points of the same job, the _metadata file size > is < 4MB. Any insights on what may cause the difference? > > Thank you! > > Regards, > -Yu >
