RE: Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Responding to this thread late but just wanted to share that I've made significant progress developing Rust implementations of Puffin reader and writer. You can view the PR here: https://github.com/apache/iceberg-rust/pull/714 Thanks, Farooq On 2024/11/22 16:25:28 Zoltán Borók-Nagy wrote: > A

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Awesome! In Impala we created our own implementations so far, but it will be nice to join forces and have a common library. Looking forward to the Slack channel. Cheers, Zoltan On Fri, Nov 22, 2024 at 5:01 PM Gang Wu wrote: > > I have created an issue [1] to collect initial ideas for the i

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

I have created an issue [1] to collect initial ideas for the iceberg-cpp project. Any feedback is appreciated. [1] https://github.com/apache/iceberg-cpp/issues/2 Best, Gang On Fri, Nov 22, 2024 at 10:11 PM Matt Topol wrote: > I will also help out with the iceberg-cpp effort, please include me

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

I will also help out with the iceberg-cpp effort, please include me on the channel. While my focus will still be on iceberg-go, I'll happily review and contribute to the C++ implementation. I do also plan on eventually implementing puffin in the iceberg-go repo lol --Matt On Fri, Nov 22, 2024 at

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

This sounds awesome. I am looking forward to the slack channel being available so I can also help! El vie, 22 nov 2024 a las 10:03, Gang Wu () escribió: > > Thanks for the support, Fokko and JB! > > Please include me in the cpp slack channel for future cooperation. > > Best, > Gang > > On Fri, Nov

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Thanks for the support, Fokko and JB! Please include me in the cpp slack channel for future cooperation. Best, Gang On Fri, Nov 22, 2024 at 4:58 PM Jean-Baptiste Onofré wrote: > Hi Gabor, > > I think it makes sense to create iceberg-cpp resources (repository, > slack channel, ...): this can ga

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hi Gabor, I think it makes sense to create iceberg-cpp resources (repository, slack channel, ...): this can gather the efforts in the Iceberg lib and Puffin implementation. Fokko can help there, he can ping me if needed (from ASF standpoint) :) Regards JB On Fri, Nov 22, 2024 at 9:25 AM Gabor K

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hey Gabor, Sorry for not replying in this thread earlier. I fully missed this thread since it was right in the middle of my prenatal leave. As already mentioned there is the Iceberg-Rust effort, and the Iceberg-Go effort is progressing nicely (I would expect to have bindings to cpp at some point)

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Thanks Gabor for the update! Yes, I think the iceberg-rust and iceberg-go are going on pretty well. I have an interest in working on a brand new iceberg-cpp project. I have done some research and found that this is a long-awaited project but has made no progress so far. I'd like to hear the voice

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hi Iceberg Community, It's been a while since we started this discussion. I'd like to revive the conversion for two reasons: 1) I think I'll have some capacity starting from early next year to take care of the C++ Puffin stuff we already talked about above, and also from the Impala community we co

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

If C++ engines prefer not to depend on Iceberg Rust, I actually don't see a problem with having a separate C++ project even if it only contains Puffin readers/writers. The important part is to avoid multiple C++ writer/reader implementations in different engines. There were concerns with having on

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hi Gabor I like the idea, it sounds good to me. Imho, for "clarity", the best option would be to have a dedicated puffin repo with different language binding (at bit like it's done in Arrow). I think that adding to iceberg-rust could be an option, but not sure it would be obvious for developers.

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hi: I think it is reasonable to start a new C++ sub-project even if it will > only contain Puffin writers/readers initially. Are there any alternatives? > We could add a separate repo for Puffin but we previously concluded it is > not a good idea to support multiple languages in one repo. I'm no

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

I definitely support the idea of having Puffin readers/writers for different languages managed by the community. It would be really unfortunate for engines to re-implement this logic. Like Piotr said, it is not only for V3 position deletes but also for table stats supported today. I think it is re

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hi, It won't be an issue if there is already an iceberg-cpp implementation. However, it is unfortunate to see duplicate efforts from different query engines to implement their own C++ Iceberg reader and writers. Is it a good chance to add official C++ implementation by providing a puffin reader/wr

Re: [DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hi Gabor, thanks for starting this topic. it would be awesome to have Puffin readers/writers available to all languages supported by the Iceberg community! The topic is important for v3, but also if we want to support stats updates when writing to tables that already have some stats collected. i

[DISCUSS] Additional language implementations for Iceberg Puffin reader/writer

Hi Iceberg Community, With the V3 position delete proposal it came up that non-Java engines might have to implement a Puffin reader and writer themselves so that they can support the newly proposed position deletes.