Thanks for driving this Sung, this is very exciting! 1. The transforms are a good first thing to address. 2. I agree with Xuanwo, that for flexibility we can decouple them. 3. Automation is probably easier than doing it manually (otherwise we would have to document the steps).
Kind regards, Fokko Op di 27 aug 2024 om 17:50 schreef Xuanwo <xua...@apache.org>: > > Thanks a lot for carrying this! > > 1. Which features do we expose in the first release? > > > No further comments on this. Pyiceberg is currently the sole user of > pyiceberg_core. I'm willing to meet whatever pyiceberg's needs. > > 2. Should the version of the pyiceberg_core python package be aligned with > the version of iceberg-rust crate? > > > I believe pyiceberg_core should have its own version. The release workflow > should be very different from the existing Rust workflow, so we don't need to > align the versions just for this. > > 3. We could publish the initial package manually to unblock PyIceberg, but we > should invest in a unified release process on github workflows that publishes > all enabled language bindings in the repository together. > > > Releasing a Rust-core-based Python package manually is very tedious; it's > better to start with a GitHub workflow from the first version. > > I have experience in building OpenDAL Python workflows, and I'm willing to > help establish the release workflow for PyIceberg Core. > > See: > https://github.com/apache/opendal/blob/main/.github/workflows/release_python.yml > > > On Tue, Aug 27, 2024, at 22:56, Sung Yun wrote: > > Hi folks, > > We’ve recently had a few discussion threads on components we wanted to expose > as Python Bindings from the iceberg-rust project in the form of the > "pyiceberg_core" package. > > Now that the Python bindings for the Transforms have been implemented, I > wanted to open this thread to discuss the first 'pyiceberg_core' release. > Here are some questions I wanted to discuss with the community: > > 1. Which features do we expose in the first release? > 2. Should the version of the pyiceberg_core python package be aligned with > the version of iceberg-rust crate? > 3. What is the release cadence we want to settle on in the future? > > Here are my answers to the above questions, where my thoughts on the > versioning and release process are inspired by the current practice from > Apache Arrow community <https://arrow.apache.org/release/>: > 1. Expose Transforms as Python bindings is ready to release, and we could > group in other features if they are also ready: > https://github.com/apache/iceberg-rust/pull/556 > 2. I think we could start with the current rust version (0.3.0). Aligning it > with the version of the iceberg-rust crate would make it easier to set up a > CI worklow that releases packages for all available bindings from the same > trigger. > 3. We could publish the initial package manually to unblock PyIceberg, but we > should invest in a unified release process on github workflows that publishes > all enabled language bindings in the repository together. > > I'm excited to hear your thoughts! > > Sung > > Prior Discussion Threads: > Transforms: https://lists.apache.org/thread/33c0nkc3k6646lvro1lv22pvhwlp50ss > FileIO: https://lists.apache.org/thread/86zotqs1wqxojt4zx8np29q5doj1l1wc > > PR for exposing transforms as Python bindings: > https://github.com/apache/iceberg-rust/pull/556 > > Xuanwo > > https://xuanwo.io/ >
