+1 for deprecating and removing this. I don't think it makes sense to ask users to pass this while we can infer it from existing arguments.
Also I see java code <https://github.com/apache/iceberg/blob/821aec32bb9be28d9c1905f772d9e3101cc98d9e/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1085> has comments to remove this. Since this is a public api, should we annotate it as deprecated in 1.x, and remove it in 2.0? On Sat, Nov 16, 2024 at 3:11 AM Kevin Liu <kevin.jq....@gmail.com> wrote: > Thanks for bringing this up Fokko. > It makes sense to hide `last-column-id` from the public API, as it is an > implementation detail. > > As mentioned in the PR, I checked references to `last-column-id` > <https://grep.app/search?current=4&q=last-column-id&filter[lang][0]=Python&filter[lang][1]=Rust&filter[lang][2]=Java&filter[lang][3]=Go&filter[lang][4]=C%2B%2B> > and `last_column_id` <https://grep.app/search?q=last_column_id> and > didn't find anything that would break due to this change. > > We would likely need to also deprecate this in PyIceberg as well. > > https://github.com/apache/iceberg-python/blob/b2f0a9e5cd7dd548e19cdcdd7f9205f03454369a/pyiceberg/table/update/__init__.py#L90-L91 > > Best, > Kevin Liu > > > On Thu, Nov 14, 2024 at 1:13 AM Fokko Driesprong <fo...@apache.org> wrote: > >> Hi everyone, >> >> While reviewing the TableMetadataBuilder PR on Iceberg-Rust >> <https://github.com/apache/iceberg-rust/pull/587#discussion_r1834400220> >> the other day, I noticed that it exposes the last-column-id to the public >> API, but I believe there is no need for it. This field is used to determine >> the next field-id when adding new fields to a schema. The last-column-id was >> added to the REST spec <https://github.com/apache/iceberg/pull/7445> a >> while ago, to make the spec in line with the reference implementation, but >> in hindsight, it should have been the other way around. >> >> My suggestion is to deprecate and remove this field from the spec and >> code <https://github.com/apache/iceberg/pull/11514/>, as I can't think >> of any use case where you want to make jumps in the last-column-id (it has >> to be monotonically increasing). This will help clean up the APIs and the >> reference implementation. >> >> Would love to hear everyone's thoughts on this! >> >> Kind regards, >> Fokko >> >> >>
