Re: [DISCUSS] Flink range partitioner/shuffle

2023-06-03 Thread Steven Wu
Ryan, thanks a lot for the feedback. Will use `StructType` when applicable. `PartitionKey` is a combination of `StructProjection` and `StructTransformation` with a flattened array of partition tuples. This pattern of flattened arrays can also work for the SortOrder purpose. But it is not the `Stru

Re: Iceberg old partition gc

2023-06-03 Thread Szehon Ho
> > @Szehon, I am wondering if we can create materialized views for metadata > tables to support infinite history on metadata tables (like snapshots or > partitions). Obviously, materialized views can't be used for time travel or > rollback. They are only meant for maintaining long/infinite histori