I think this is a good idea but I have concerns about compatibility. IMO, I think changing the cardinality of input columns is a large enough change that trying to retrofit it into V1 or V2 of the specification will cause pain for implementations not relying on reference implementation. I
As a secondary concern, I think it would be worthwhile for PMC to formalize the process around specification changes as these have broader implications for Iceberg adoption. A model that I've seen work reasonably well in other communities is the following: 1. Discussion of overall features on the mailing list (this can also be a pointer to the GitHub issue). 2. 2 reference implementations demonstrating the change is viable (it seems like PyIceberg is close to being fully functional enough that this will be viable in the near term). 3. A formal vote adopting the change. But really any statement of policy around how specification changes occur (and what changes will be considered for backporting to finalized specifications) would be useful. Thanks, Micah On Sat, Jan 27, 2024 at 2:55 AM 叶先进 <advance...@gmail.com> wrote: > Hi, > > This is just a heads up. Szehon and I just make a spec change to include > multi-arg transform: https://github.com/apache/iceberg/pull/8579 recently. > I am sending this to get input from others who did not review the pr before > Iceberg 1.5 release. Any concerns/suggestions are appreciated. > > After this change, we are working to get the API/Core and engine changes > into the iceberg and more importantly the concrete multi-arg transforms, > such as bucketV2 or zorder, etc. >
