Thanks a lot for carrying this! > 1. Which features do we expose in the first release?
No further comments on this. Pyiceberg is currently the sole user of pyiceberg_core. I'm willing to meet whatever pyiceberg's needs. > 2. Should the version of the pyiceberg_core python package be aligned with > the version of iceberg-rust crate? I believe pyiceberg_core should have its own version. The release workflow should be very different from the existing Rust workflow, so we don't need to align the versions just for this. > 3. We could publish the initial package manually to unblock PyIceberg, but we > should invest in a unified release process on github workflows that publishes > all enabled language bindings in the repository together. Releasing a Rust-core-based Python package manually is very tedious; it's better to start with a GitHub workflow from the first version. I have experience in building OpenDAL Python workflows, and I'm willing to help establish the release workflow for PyIceberg Core. See: https://github.com/apache/opendal/blob/main/.github/workflows/release_python.yml On Tue, Aug 27, 2024, at 22:56, Sung Yun wrote: > Hi folks, > > We’ve recently had a few discussion threads on components we wanted to expose > as Python Bindings from the iceberg-rust project in the form of the > "pyiceberg_core" package. > > Now that the Python bindings for the Transforms have been implemented, I > wanted to open this thread to discuss the first 'pyiceberg_core' release. > Here are some questions I wanted to discuss with the community: > > 1. Which features do we expose in the first release? > 2. Should the version of the pyiceberg_core python package be aligned with > the version of iceberg-rust crate? > 3. What is the release cadence we want to settle on in the future? > > Here are my answers to the above questions, where my thoughts on the > versioning and release process are inspired by the current practice from > Apache Arrow community <https://arrow.apache.org/release/>: > 1. Expose Transforms as Python bindings is ready to release, and we could > group in other features if they are also ready: > https://github.com/apache/iceberg-rust/pull/556 > 2. I think we could start with the current rust version (0.3.0). Aligning it > with the version of the iceberg-rust crate would make it easier to set up a > CI worklow that releases packages for all available bindings from the same > trigger. > 3. We could publish the initial package manually to unblock PyIceberg, but we > should invest in a unified release process on github workflows that publishes > all enabled language bindings in the repository together. > > I'm excited to hear your thoughts! > > Sung > > Prior Discussion Threads: > Transforms: https://lists.apache.org/thread/33c0nkc3k6646lvro1lv22pvhwlp50ss > FileIO: https://lists.apache.org/thread/86zotqs1wqxojt4zx8np29q5doj1l1wc > > PR for exposing transforms as Python bindings: > https://github.com/apache/iceberg-rust/pull/556 Xuanwo https://xuanwo.io/
