‘If would’ → ‘it would’ ‘original schema’ → ‘original scheme’
On Tue, 8 Oct 2019 at 18:00, Elliot West <tea...@gmail.com> wrote: > Hello, > > I'm trying to understand the underlying partitioning model in Iceberg. It > is not clear to me how partition keys are distributed with respect to > actual files and what constraints exist for partition evolution. My > expectation is that to achieve reasonable read performance, sets of keys > must be assigned to specific files so that partition pruning can be > effective. If would then follow that later evolutions of partitioning > schemes must be derived only from the original schema and therefore, they > must effectively be a coarser grained rollup (i.e. a year from a date). > > Is this correct? I'm unable to discern this explicitly from the > documentation as it doesn't mention constraints and it could perhaps be > over eagerly be interpreted as describing a panacea: > > Most importantly, queries no longer depend on a table’s physical layout. >> With a separation between physical and logical, Iceberg tables can evolve >> partition schemes over time as data volume changes. Misconfigured tables >> can be fixed without an expensive migration. > > https://iceberg.apache.org/partitioning/#icebergs-hidden-partitioning > > Thanks for your time, > > Elliot. >