Sorry I may have misunderstood the statement and maybe this is specific to
multi-arg transform, in any case let's get a spec pr earlier in to
discuss/specify behavior for V1-2 vs 3.
Thanks
Szehon
On Tue, Jan 30, 2024 at 9:23 AM Szehon Ho wrote:
> Thanks all for the discussion.
>
> For the speci
Thanks all for the discussion.
For the specific point about any new transform being able to be read in
current versions but only written in V3 (which I missed as well):
While this is a v3 feature and must be supported for v3 compatibility, the
> community usually also has guidelines for using fea
Thanks YE, Ryan and Szehon for your thoughts.
As was already touched on, my primary concern is it seems like features
were being added to V2 that were not forward compatible. It seems there is
consensus that these will be V3 and possibly backported to V2, this
makes much more sense. I've added s
Sorry for the previous email, it was sent by accident without complete
reply. Please discard the previous email, and see the whole reply in this
email.
Thanks for Micah and Ryan's reply.
As Szehon already pointed out, this change is to allow creation of
*new* multi-arg
transforms. I remember ther
Thanks for Micah and Ryan's reply.
As Szehon already pointed out, this change is to allow creation of *new*
multi-arg transforms. I remember there's a discussion in the google doc
whether targeting this as a `V3` spec change, it turns out that we may
support this as long as we make sure old writer
Thanks for working on this, Szehon and AdvanceXY! I'm glad to see this
picking up for the v3 work.
I also want to address Micah's comments and suggest how we can do better
next time. From Micah's suggestion, there are 3 steps: 1. Discuss the
feature, 2. Build 2 reference implementations, and 3. ho
Hi,
This would not be retrofitting existing partition transforms, but just
allowing for the creation of new multi-arg transforms. Is the concern that
some implementations are never expecting new transforms to be added? Old
implementations would indeed not be able to read Iceberg tables created
w
I think this is a good idea but I have concerns about compatibility. IMO, I
think changing the cardinality of input columns is a large enough change
that trying to retrofit it into V1 or V2 of the specification will cause
pain for implementations not relying on reference implementation. I
As a se
Hi,
This is just a heads up. Szehon and I just make a spec change to include
multi-arg transform: https://github.com/apache/iceberg/pull/8579 recently. I am
sending this to get input from others who did not review the pr before Iceberg
1.5 release. Any concerns/suggestions are appreciated.
Aft
