I will also help out with the iceberg-cpp effort, please include me on the channel. While my focus will still be on iceberg-go, I'll happily review and contribute to the C++ implementation.
I do also plan on eventually implementing puffin in the iceberg-go repo lol --Matt On Fri, Nov 22, 2024 at 8:55 AM Raúl Cumplido <rau...@apache.org> wrote: > This sounds awesome. I am looking forward to the slack channel being > available so I can also help! > > El vie, 22 nov 2024 a las 10:03, Gang Wu (<ust...@gmail.com>) escribió: > > > > Thanks for the support, Fokko and JB! > > > > Please include me in the cpp slack channel for future cooperation. > > > > Best, > > Gang > > > > On Fri, Nov 22, 2024 at 4:58 PM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >> > >> Hi Gabor, > >> > >> I think it makes sense to create iceberg-cpp resources (repository, > >> slack channel, ...): this can gather the efforts in the Iceberg lib > >> and Puffin implementation. > >> > >> Fokko can help there, he can ping me if needed (from ASF standpoint) :) > >> > >> Regards > >> JB > >> > >> On Fri, Nov 22, 2024 at 9:25 AM Gabor Kaszab <gaborkas...@apache.org> > wrote: > >> > > >> > Hi Iceberg Community, > >> > > >> > It's been a while since we started this discussion. I'd like to > revive the conversion for two reasons: > >> > 1) I think I'll have some capacity starting from early next year to > take care of the C++ Puffin stuff we already talked about above, and also > from the Impala community we could add some additional auxiliary > functionality for the V3 positional deletes later on. > >> > 2) I learned that a part of the community is interested in having a > C++ implementation of the Iceberg lib in general for their C++ engine. cc > @Gang Wu > >> > > >> > There seemed to be general support from the community to start up > such a sub-project, so I'm reaching out now to ask for some guidance so > that we can get going. @Fokko Driesprong You have much experience in this > area, do you think you have the bandwidth to handhold us with the process > and the steps? Meanwhile, I'll take a look at the Python repo to get a feel > for myself. > >> > > >> > Regards, > >> > Gabor > >> > > >> > On Wed, Sep 11, 2024 at 5:32 PM Anton Okolnychyi < > aokolnyc...@gmail.com> wrote: > >> >> > >> >> If C++ engines prefer not to depend on Iceberg Rust, I actually > don't see a problem with having a separate C++ project even if it only > contains Puffin readers/writers. The important part is to avoid multiple > C++ writer/reader implementations in different engines. > >> >> > >> >> There were concerns with having one repo for multiple languages when > we discussed how to maintain the main project. We should re-evaluate if > they still apply here. > >> >> > >> >> - Anton > >> >> > >> >> вт, 10 вер. 2024 р. о 06:08 Gabor Kaszab <gaborkas...@apache.org> > пише: > >> >>> > >> >>> Hey All, > >> >>> > >> >>> Thanks for all the answers! > >> >>> > >> >>> My initial thought here was to have an iceberg-cpp repo for the > Puffin reader/writer implementation in C++. But then I thought that there > might be only one thing within this cpp repo, that is the Puffin stuff, and > once implemented the repo won't have any more PRs coming in so it might be > an overkill to introduce a separate repo for this purpose. Unless there is > a need from other stakeholders to have a C++ implementation of the Iceberg > lib (similarly to Python and Rust) but this would lead to a different > conversation. > >> >>> > >> >>> Pulling in the Rust implementation into the C++ part of Impala: I > don't know, I have to do some research on this. For performance reasons I > know Impala prefers to have its own implementation of things (like Parquet > reader/writer), so I have to double-check if it's acceptable > performance-wise to pull in a Rust implementation of anything. I don't have > any experience of doing so, hence the hesitation. > >> >>> > >> >>> JB, you mean that we can create a sub-repo like iceberg-puffin, > that would hold separate implementations for different languages? For me > this could also work. Wondering what others think. > >> >>> > >> >>> Regards, > >> >>> Gabor > >> >>> > >> >>> > >> >>> On Fri, Aug 30, 2024 at 2:08 PM Jean-Baptiste Onofré < > j...@nanthrax.net> wrote: > >> >>>> > >> >>>> Hi Gabor > >> >>>> > >> >>>> I like the idea, it sounds good to me. > >> >>>> > >> >>>> Imho, for "clarity", the best option would be to have a dedicated > >> >>>> puffin repo with different language binding (at bit like it's done > in > >> >>>> Arrow). > >> >>>> I think that adding to iceberg-rust could be an option, but not > sure > >> >>>> it would be obvious for developers. > >> >>>> > >> >>>> Regards > >> >>>> JB > >> >>>> > >> >>>> On Thu, Aug 29, 2024 at 2:46 PM Gabor Kaszab < > gaborkas...@apache.org> wrote: > >> >>>> > > >> >>>> > Hi Iceberg Community, > >> >>>> > > >> >>>> > With the V3 position delete proposal it came up that non-Java > engines might have to implement a Puffin reader and writer themselves so > that they can support the newly proposed position deletes. Impala would > most probably require a C++ implementation and first of all I'm wondering > whether there are other engines that require additional language > implementation. > >> >>>> > > >> >>>> > My additional thought is that once we have an implementation in > another language where should that live? I can do the C++ and that could > live within Impala, but I think it could be useful for other engines too > but I have no idea where such an implementation could live TBH. > >> >>>> > > >> >>>> > Would be nice to hear opinions on this! > >> >>>> > Thanks, > >> >>>> > Gabor >
