Hi Brian,
thanks for starting the discussion around a native library for iceberg.
I'm Jan the owner of the iceberg-rust repository. Recently, another
project started the implementation of a iceberg rust library
(https://github.com/icelake-io/icelake). I think this is a great timing
to start a rust project in the official apache iceberg repository that
we can all work on together.
I have also been experimenting with ways to combine a C++ and a rust
library. There a ways to create a rust library that could be used from
C++ but these would bring minor inconveniences on the rust side. I don't
know how many people would want to use a rust SDK from C++ and therefore
I'm not sure if this is worth the effort.
Another point I was thinking about is whether to include the support for
the WASM component model in the rust library. This way all wasm
compatible programming languages would have access to the iceberg SDK.
I'm curious what your opinions are on these topics.
Best wiches,
Jan
On 06.07.23 12:08, Brian Olsen wrote:
Hey Iceberg Nation,
From Reddit:
https://www.reddit.com/r/dataengineering/comments/14rcaj9/iceberg_won_the_table_format_war_but_not_in_the/jqva6if/
What I want most right now is a native library for Iceberg format,
that can be used to read/write Iceberg table from multiple programming
languages, similar to how delta-rs supports accessing Delta Lake
format in Rust/Python/etc...
https://github.com/apache/iceberg/issues/5122
This issue looks at both C++ and Rust, I’m not sure how much demand
there is for C++ but there’s already a library for Iceberg Rust with
22 stars and 7 forks.
https://github.com/JanKaul/iceberg-rust
Have we seen more demand for this? Delta Lake supports it but that
alone isn’t a great reason. At what point would it make sense to get
this adopted by the project? Would this involve gathering evidence,
presenting it, and taking a vote?
Thanks
