Xuanwo, would PyIceberg and iceberg-rust share the underlying OpenDAL implementations via pyo3 / fsspec bindings<https://github.com/apache/opendal/issues/4511>?
________________________________ From: Joe Stein <crypt...@gmail.com> Sent: Thursday, August 1, 2024 3:37 AM To: dev@iceberg.apache.org <dev@iceberg.apache.org> Subject: Re: [DISCUSS] Use iceberg-rust as pyiceberg file io CAUTION: This email originates from an external party (outside of Palantir). If you believe this message is suspicious in nature, please use the "Report Message" button built into Outlook. Kafka did this with librdkafka and was wildly successful. The underlying bindings being in rust are great with a layer for access in Python +1 ~ Joe Stein On Wed, Jul 31, 2024 at 10:29 PM Xuanwo <xua...@apache.org<mailto:xua...@apache.org>> wrote: Hello everyone I start this thread to discuss the idea about using iceberg-rust as pyiceberg file io. The idea is living at https://hackmd.io/@xuanwo/iceberg_rust_as_file_io [hackmd.io]<https://urldefense.com/v3/__https://hackmd.io/@xuanwo/iceberg_rust_as_file_io__;!!NkS9JGVQ2sDq!7Js41FIzh2smsAOySXrKd527DXCmXdrwV8Uov8TIdQqLRcsCkfPnHzfbxbX_xctpoNpYw2XGfrduTPd6ppTI$> In summary, we can leverage the work from iceberg-rust to help pyiceberg in developing a fast and compact file IO system that benefits users with specific constraints. Welcome to join in the discussion. Xuanwo https://xuanwo.io/ [xuanwo.io]<https://urldefense.com/v3/__https://xuanwo.io/__;!!NkS9JGVQ2sDq!7Js41FIzh2smsAOySXrKd527DXCmXdrwV8Uov8TIdQqLRcsCkfPnHzfbxbX_xctpoNpYw2XGfrduTNspr1jI$>
