Right now, both should be guaranteed. The library will only create a new spec identifier if there isn't an existing spec that meets the second equality definition. That's implemented here: https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1537-L1549
The spec assumes that no one would create the same spec with different IDs, but we should probably clarify that it isn't allowed. For how this applies to tracking delete files, it is the partition spec ID that should be checked. On Fri, Nov 3, 2023 at 10:35 AM Micah Kornfield <emkornfi...@gmail.com> wrote: > Hello Iceberg Dev, > The Iceberg specification for matching delete files with data files during > scan planning states : > > "The data file’s partition (both spec and partition values) is equal to > the delete file’s partition" > > Equality of partition specs appears slightly ambiguous (apologies if I > missed this). I can imagine two different definitions: > 1. Equality of partition spec identifier. > 2. Equality of each element in the partition spec tuple (which raises the > further question of which elements of a partition transform are considered > for equality). This is distinct from option one only when identical tuples > are allowed in the partition spec map with two different identifiers. > > I would assume the definition should be based on the partition spec > identifier but wanted to clarify. > > Thanks, > Micah > > > > > > > > -- Ryan Blue Tabular
