Re: Spec change for multi-arg transform

2024-01-28 Thread YE
Sorry for the previous email, it was sent by accident without complete reply. Please discard the previous email, and see the whole reply in this email. Thanks for Micah and Ryan's reply. As Szehon already pointed out, this change is to allow creation of *new* multi-arg transforms. I remember ther

Re: Spec change for multi-arg transform

2024-01-28 Thread YE
Thanks for Micah and Ryan's reply. As Szehon already pointed out, this change is to allow creation of *new* multi-arg transforms. I remember there's a discussion in the google doc whether targeting this as a `V3` spec change, it turns out that we may support this as long as we make sure old writer

Re: [DISCUSS] PyIceberg 0.6.0 release

2024-01-28 Thread Honah J.
Really excited for the upcoming 0.6.0 release and its new features! Big thanks to everyone for their hard work. I'm looking forward to the community feedback and future enhancements. Best regards, Honah On Fri, Jan 26, 2024 at 1:56 PM Daniel Weeks wrote: > I'm also strongly in favor of getting

Re: Spec change for multi-arg transform

2024-01-28 Thread Ryan Blue
Thanks for working on this, Szehon and AdvanceXY! I'm glad to see this picking up for the v3 work. I also want to address Micah's comments and suggest how we can do better next time. From Micah's suggestion, there are 3 steps: 1. Discuss the feature, 2. Build 2 reference implementations, and 3. ho

Re: Spec change for multi-arg transform

2024-01-28 Thread Szehon Ho
Hi, This would not be retrofitting existing partition transforms, but just allowing for the creation of new multi-arg transforms. Is the concern that some implementations are never expecting new transforms to be added? Old implementations would indeed not be able to read Iceberg tables created w