I have created an issue [1] to collect initial ideas for the iceberg-cpp project.
Any feedback is appreciated. [1] https://github.com/apache/iceberg-cpp/issues/2 Best, Gang On Fri, Nov 22, 2024 at 10:11 PM Matt Topol <zotthewiz...@gmail.com> wrote: > I will also help out with the iceberg-cpp effort, please include me on the > channel. While my focus will still be on iceberg-go, I'll happily review > and contribute to the C++ implementation. > > I do also plan on eventually implementing puffin in the iceberg-go repo lol > > --Matt > > On Fri, Nov 22, 2024 at 8:55 AM Raúl Cumplido <rau...@apache.org> wrote: > >> This sounds awesome. I am looking forward to the slack channel being >> available so I can also help! >> >> El vie, 22 nov 2024 a las 10:03, Gang Wu (<ust...@gmail.com>) escribió: >> > >> > Thanks for the support, Fokko and JB! >> > >> > Please include me in the cpp slack channel for future cooperation. >> > >> > Best, >> > Gang >> > >> > On Fri, Nov 22, 2024 at 4:58 PM Jean-Baptiste Onofré <j...@nanthrax.net> >> wrote: >> >> >> >> Hi Gabor, >> >> >> >> I think it makes sense to create iceberg-cpp resources (repository, >> >> slack channel, ...): this can gather the efforts in the Iceberg lib >> >> and Puffin implementation. >> >> >> >> Fokko can help there, he can ping me if needed (from ASF standpoint) :) >> >> >> >> Regards >> >> JB >> >> >> >> On Fri, Nov 22, 2024 at 9:25 AM Gabor Kaszab <gaborkas...@apache.org> >> wrote: >> >> > >> >> > Hi Iceberg Community, >> >> > >> >> > It's been a while since we started this discussion. I'd like to >> revive the conversion for two reasons: >> >> > 1) I think I'll have some capacity starting from early next year to >> take care of the C++ Puffin stuff we already talked about above, and also >> from the Impala community we could add some additional auxiliary >> functionality for the V3 positional deletes later on. >> >> > 2) I learned that a part of the community is interested in having a >> C++ implementation of the Iceberg lib in general for their C++ engine. cc >> @Gang Wu >> >> > >> >> > There seemed to be general support from the community to start up >> such a sub-project, so I'm reaching out now to ask for some guidance so >> that we can get going. @Fokko Driesprong You have much experience in this >> area, do you think you have the bandwidth to handhold us with the process >> and the steps? Meanwhile, I'll take a look at the Python repo to get a feel >> for myself. >> >> > >> >> > Regards, >> >> > Gabor >> >> > >> >> > On Wed, Sep 11, 2024 at 5:32 PM Anton Okolnychyi < >> aokolnyc...@gmail.com> wrote: >> >> >> >> >> >> If C++ engines prefer not to depend on Iceberg Rust, I actually >> don't see a problem with having a separate C++ project even if it only >> contains Puffin readers/writers. The important part is to avoid multiple >> C++ writer/reader implementations in different engines. >> >> >> >> >> >> There were concerns with having one repo for multiple languages >> when we discussed how to maintain the main project. We should re-evaluate >> if they still apply here. >> >> >> >> >> >> - Anton >> >> >> >> >> >> вт, 10 вер. 2024 р. о 06:08 Gabor Kaszab <gaborkas...@apache.org> >> пише: >> >> >>> >> >> >>> Hey All, >> >> >>> >> >> >>> Thanks for all the answers! >> >> >>> >> >> >>> My initial thought here was to have an iceberg-cpp repo for the >> Puffin reader/writer implementation in C++. But then I thought that there >> might be only one thing within this cpp repo, that is the Puffin stuff, and >> once implemented the repo won't have any more PRs coming in so it might be >> an overkill to introduce a separate repo for this purpose. Unless there is >> a need from other stakeholders to have a C++ implementation of the Iceberg >> lib (similarly to Python and Rust) but this would lead to a different >> conversation. >> >> >>> >> >> >>> Pulling in the Rust implementation into the C++ part of Impala: I >> don't know, I have to do some research on this. For performance reasons I >> know Impala prefers to have its own implementation of things (like Parquet >> reader/writer), so I have to double-check if it's acceptable >> performance-wise to pull in a Rust implementation of anything. I don't have >> any experience of doing so, hence the hesitation. >> >> >>> >> >> >>> JB, you mean that we can create a sub-repo like iceberg-puffin, >> that would hold separate implementations for different languages? For me >> this could also work. Wondering what others think. >> >> >>> >> >> >>> Regards, >> >> >>> Gabor >> >> >>> >> >> >>> >> >> >>> On Fri, Aug 30, 2024 at 2:08 PM Jean-Baptiste Onofré < >> j...@nanthrax.net> wrote: >> >> >>>> >> >> >>>> Hi Gabor >> >> >>>> >> >> >>>> I like the idea, it sounds good to me. >> >> >>>> >> >> >>>> Imho, for "clarity", the best option would be to have a dedicated >> >> >>>> puffin repo with different language binding (at bit like it's >> done in >> >> >>>> Arrow). >> >> >>>> I think that adding to iceberg-rust could be an option, but not >> sure >> >> >>>> it would be obvious for developers. >> >> >>>> >> >> >>>> Regards >> >> >>>> JB >> >> >>>> >> >> >>>> On Thu, Aug 29, 2024 at 2:46 PM Gabor Kaszab < >> gaborkas...@apache.org> wrote: >> >> >>>> > >> >> >>>> > Hi Iceberg Community, >> >> >>>> > >> >> >>>> > With the V3 position delete proposal it came up that non-Java >> engines might have to implement a Puffin reader and writer themselves so >> that they can support the newly proposed position deletes. Impala would >> most probably require a C++ implementation and first of all I'm wondering >> whether there are other engines that require additional language >> implementation. >> >> >>>> > >> >> >>>> > My additional thought is that once we have an implementation in >> another language where should that live? I can do the C++ and that could >> live within Impala, but I think it could be useful for other engines too >> but I have no idea where such an implementation could live TBH. >> >> >>>> > >> >> >>>> > Would be nice to hear opinions on this! >> >> >>>> > Thanks, >> >> >>>> > Gabor >> >
