Hey Gabor, Sorry for not replying in this thread earlier. I fully missed this thread since it was right in the middle of my prenatal leave.
As already mentioned there is the Iceberg-Rust effort, and the Iceberg-Go effort is progressing nicely (I would expect to have bindings to cpp at some point). However, I think there is room for having a dedicated cpp implementation to not push developers in a certain direction. Looking at the interest here, I went ahead and created the iceberg-cpp <https://github.com/apache/iceberg-cpp> repository. I'll requested a cpp slack channel as well, where we can sync up and make this happen. Looking forward to this! Kind regards, Fokko Op vr 22 nov 2024 om 09:36 schreef Gang Wu <ust...@gmail.com>: > Thanks Gabor for the update! > > Yes, I think the iceberg-rust and iceberg-go are going on pretty well. I > have an interest in working on a brand new iceberg-cpp project. > > I have done some research and found that this is a long-awaited project > but has made no progress so far. I'd like to hear the voice from the > community to see if it is something that is worth doing. > > Best, > Gang > > On Fri, Nov 22, 2024 at 4:26 PM Gabor Kaszab <gaborkas...@apache.org> > wrote: > >> Hi Iceberg Community, >> >> It's been a while since we started this discussion. I'd like to revive >> the conversion for two reasons: >> 1) I think I'll have some capacity starting from early next year to take >> care of the C++ Puffin stuff we already talked about above, and also from >> the Impala community we could add some additional auxiliary functionality >> for the V3 positional deletes later on. >> 2) I learned that a part of the community is interested in having a C++ >> implementation of the Iceberg lib in general for their C++ engine. cc @Gang >> Wu >> >> There seemed to be general support from the community to start up such a >> sub-project, so I'm reaching out now to ask for some guidance so that we >> can get going. @Fokko Driesprong <fo...@apache.org> You have much >> experience in this area, do you think you have the bandwidth to handhold us >> with the process and the steps? Meanwhile, I'll take a look at the Python >> repo to get a feel for myself. >> >> Regards, >> Gabor >> >> On Wed, Sep 11, 2024 at 5:32 PM Anton Okolnychyi <aokolnyc...@gmail.com> >> wrote: >> >>> If C++ engines prefer not to depend on Iceberg Rust, I actually don't >>> see a problem with having a separate C++ project even if it only contains >>> Puffin readers/writers. The important part is to avoid multiple C++ >>> writer/reader implementations in different engines. >>> >>> There were concerns with having one repo for multiple languages when we >>> discussed how to maintain the main project. We should re-evaluate if they >>> still apply here. >>> >>> - Anton >>> >>> вт, 10 вер. 2024 р. о 06:08 Gabor Kaszab <gaborkas...@apache.org> пише: >>> >>>> Hey All, >>>> >>>> Thanks for all the answers! >>>> >>>> My initial thought here was to have an iceberg-cpp repo for the Puffin >>>> reader/writer implementation in C++. But then I thought that there might be >>>> only one thing within this cpp repo, that is the Puffin stuff, and once >>>> implemented the repo won't have any more PRs coming in so it might be an >>>> overkill to introduce a separate repo for this purpose. Unless there is a >>>> need from other stakeholders to have a C++ implementation of the Iceberg >>>> lib (similarly to Python and Rust) but this would lead to a different >>>> conversation. >>>> >>>> Pulling in the Rust implementation into the C++ part of Impala: I don't >>>> know, I have to do some research on this. For performance reasons I know >>>> Impala prefers to have its own implementation of things (like Parquet >>>> reader/writer), so I have to double-check if it's acceptable >>>> performance-wise to pull in a Rust implementation of anything. I don't have >>>> any experience of doing so, hence the hesitation. >>>> >>>> JB, you mean that we can create a sub-repo like iceberg-puffin, that >>>> would hold separate implementations for different languages? For me this >>>> could also work. Wondering what others think. >>>> >>>> Regards, >>>> Gabor >>>> >>>> >>>> On Fri, Aug 30, 2024 at 2:08 PM Jean-Baptiste Onofré <j...@nanthrax.net> >>>> wrote: >>>> >>>>> Hi Gabor >>>>> >>>>> I like the idea, it sounds good to me. >>>>> >>>>> Imho, for "clarity", the best option would be to have a dedicated >>>>> puffin repo with different language binding (at bit like it's done in >>>>> Arrow). >>>>> I think that adding to iceberg-rust could be an option, but not sure >>>>> it would be obvious for developers. >>>>> >>>>> Regards >>>>> JB >>>>> >>>>> On Thu, Aug 29, 2024 at 2:46 PM Gabor Kaszab <gaborkas...@apache.org> >>>>> wrote: >>>>> > >>>>> > Hi Iceberg Community, >>>>> > >>>>> > With the V3 position delete proposal it came up that non-Java >>>>> engines might have to implement a Puffin reader and writer themselves so >>>>> that they can support the newly proposed position deletes. Impala would >>>>> most probably require a C++ implementation and first of all I'm wondering >>>>> whether there are other engines that require additional language >>>>> implementation. >>>>> > >>>>> > My additional thought is that once we have an implementation in >>>>> another language where should that live? I can do the C++ and that could >>>>> live within Impala, but I think it could be useful for other engines too >>>>> but I have no idea where such an implementation could live TBH. >>>>> > >>>>> > Would be nice to hear opinions on this! >>>>> > Thanks, >>>>> > Gabor >>>>> >>>>
