Hi Chris, Thanks for raising this. Do you know how big the Ruby data community is? I think the most important part is that it gets some traction and will continue to be maintained.
I fully agree that building on top of iceberg-rust makes a lot of sense, since also with PyIceberg we're running into limitations when it comes to performance and limited parallelism. Kind regards, Fokko Op ma 5 aug 2024 om 14:14 schreef Xuanwo <xua...@apache.org>: > Hi, Chris > > I love this idea. One of the main reasons I started working on > iceberg-rust is due to the potential that a rust-powered iceberg core can > offer. > > I'm not an experienced ruby developer, but I'm willing to help with some > CI setup or docs since I have some experience in the opendal community with > ruby bindings. > > On Mon, Aug 5, 2024, at 20:03, Renjie Liu wrote: > > Hi, Chris: > > Thanks for raising this. Generally I'm +1 with building ruby bindings on > top of rust implementation, who would help introduce iceberg into the ruby > ecosystem. > > On Mon, Aug 5, 2024 at 7:30 PM Chris Atkins <chri...@buildkite.com.invalid> > wrote: > > Hi there, > > I'm following up on a discussion > <https://apache-iceberg.slack.com/archives/C05HTENMJG4/p1722750831522969> from > the #rust channel on the Iceberg community slack, so starting a thread here > too. > > After seeing Xuanwo's and Song's recent proposals around leveraging > iceberg-rust to power parts of PyIceberg, I was thinking it could be > valuable to follow a similar pattern to build out Ruby bindings for > Iceberg. Being able to stand on the shoulders of iceberg-rust could really > help build out a robust Ruby interface, and also offer some opportunities > for interop with things like datafusion and opendal. > > Recently in the Ruby ecosystem, writing native extensions in Rust has > become more popular, and tools like rb-sys and magnus provide a lot of the > required infrastructure. A good example is ruby-polars, which provides an > interface that is idiomatic Ruby but retains good symmetry with the APIs > exposed by py-polars. I wonder if we could eventually aim for a similar > type of symmetry between PyIceberg and a Ruby gem? > > Is there much interest in this? I've started playing around with some of > the basics, and started out with a plain native Ruby implementation of some > of the basic metadata APIs, but quickly realised that building on > iceberg-rust could be more productive than writing it all from scratch. > > *References* > > https://lists.apache.org/thread/5570vbdkrk7mdswt4jqy45lv7y58pz4b > https://lists.apache.org/thread/33c0nkc3k6646lvro1lv22pvhwlp50ss > https://github.com/apache/iceberg-rust/pull/518 > > *Prior Art in Ruby* > > https://github.com/matsadler/magnus > https://github.com/oxidize-rb/rb-sys > https://github.com/ankane/ruby-polars > https://github.com/apache/opendal/tree/main/bindings/ruby > > Thanks, > Chris Atkins > > Xuanwo > > https://xuanwo.io/ > >
