Hello, I'm trying to understand the underlying partitioning model in Iceberg. It is not clear to me how partition keys are distributed with respect to actual files and what constraints exist for partition evolution. My expectation is that to achieve reasonable read performance, sets of keys must be assigned to specific files so that partition pruning can be effective. If would then follow that later evolutions of partitioning schemes must be derived only from the original schema and therefore, they must effectively be a coarser grained rollup (i.e. a year from a date).
Is this correct? I'm unable to discern this explicitly from the documentation as it doesn't mention constraints and it could perhaps be over eagerly be interpreted as describing a panacea: Most importantly, queries no longer depend on a table’s physical layout. > With a separation between physical and logical, Iceberg tables can evolve > partition schemes over time as data volume changes. Misconfigured tables > can be fixed without an expensive migration. https://iceberg.apache.org/partitioning/#icebergs-hidden-partitioning Thanks for your time, Elliot.