Peter, Iceberg keeps track of the last few metadata files and cleans them up after it reaches a configurable max, which I think is defaulted to 100.
On Tue, Jun 15, 2021 at 5:58 PM Peter Giles <gil...@uw.edu> wrote: > I'm exploring the use of Iceberg (0.11.1) in a spark app, and I'm very > enthusiastic about the features, but I am seeing something unexpected in my > testing: there is an ever increasing accumulation of v#.metadata.json files > for my test table, and I am not seeing them get cleaned up when using any > of these procedures: > > expire_snapshots > rewrite_manifests > remove_orphan_files > > Is this expected? Is there a process for pruning this set of files to > just what is needed to support the remaining snapshots? > > Thank you, > Peter > -- Ryan Blue
