[DISCUSS] Dropping Spark 2.4 support

on 2021-05-17 and it also looks like the 2.4 branch on the Spark Github repository is stale, so I don't expect any further releases. Before creating a PR I would like to check on the mail-list if anyone has any objections. If so, please let us know. Thanks, Fokko Driesprong

Re: Re: C++/Rust SDK sync

ails to invite later this week. >> > >>>> >> > >>>> Cheers, >> > >>>> Jack Ye >> > >>>> >> > >>>> On Fri, Apr 7, 2023 at 5:47 AM Driesprong, Fokko >> wrote: >> > >>>>>

Re: [DISCUSS] Spark 3.1 support?

Spring cleaning! I checked which versions of Spark the cloud vendors are supporting. Both AWS and GCP are already on 3.3. However, for Azure , Spark 3.3 is in preview and is still on 3.1.3. They are planning to upg

Re: What is the harm of adding partition to iceberg table？

Hi ZC C, Adding partitions to Iceberg tables is easy, and changing them, later on, is easy as well. The existing data will continue to exist with the partition that it was initially written with, new data will be written according to the active partitioning. When you rewrite the data (for example

Re: Welcome new committers and PMC!

Fantastic! Great having you all aboard. Cheers, Fokko Op do 4 mei 2023 om 07:40 schreef Gidon Gershinsky : > Congratulations Amogh, Eduard, Szehon! > > Cheers, Gidon > > > On Thu, May 4, 2023 at 7:59 AM Péter Váry > wrote: > >> Congratulations everyone! >> Well deserved! >> >> On Thu, May 4, 20

Re: Orphan files

Hey Gaurav, Orphan files do not affect Iceberg's performance, since Iceberg performs no list operations. It will only increase your storage bill since you have files around that are not relevant anymore. iceberg tables do need periodic maintenance, for example, it is good to rewrite small files

Re: [VOTE] Release Apache Iceberg 1.3.0 RC0

+1 (binding) Thanks for running this Anton! - Checked the signature and checksum - Checked the licenses - Build with JDK 11 - Tested against Trino Kind regards, Fokko Op do 25 mei 2023 om 05:18 schreef Ajantha Bhat : > +1 (non-binding) > > - Verifi

[VOTE] Release PyIceberg 0.4.0 RC1

Hi Everyone, Excited to start the 0.4.0 PyIceberg release process. The 0.4.0 release is packed with cool features: - Support for converting Parquet schemas into Iceberg ones - Support for reading data using FSSpec

[VOTE] Release PyIceberg 0.4.0 RC2

All, Excited to start the 0.4.0 PyIceberg release process. The 0.4.0 release is packed with awesome features: - Support for converting Parquet schemas into Iceberg ones - Support for reading data using FSSpec

Re: [VOTE] Release PyIceberg 0.4.0 RC2

Thanks all! The vote has passed: +1: Ryan Blue (binding) Jean-Baptiste Onofré (non-binding) Jack Ye (binding) Daniel Weeks (binding) Jonas Jiang (non-binding) Eduard Tutenhoefner (non-binding) Fokko Driesprong (binding) +/-0: ∅ -1: ∅ Thanks to everyone voting, and I'll publish the artifac

[ANNOUNCE] Apache PyIceberg release 0.4.0

Hi everyone! I'm pleased to announce the release of Apache PyIceberg 0.4.0! Apache Iceberg is an open table format for huge analytic datasets. Iceberg delivers high query performance for tables with tens of petabytes of data, along with atomic commits, concurrent writes, and SQL-compatible table

Re: [DISCUSS] Apache Iceberg Release 1.3.1

Hi Szehon, Thank you for the updates. I'm in favor of 1.3.1 as well. I got notified of a discrepancy in Java's TableMetadata reader today. I have a fix here against the master branch. Once that is in, I thi

Re: [VOTE] Release Apache Iceberg 1.3.1 RC1

Hi Szehon, +1 (binding) - Checked the signature and hash - Ran the RAT checks - Did a local build and ran the tests (all passed, except TestS3RestSigner tests, since #7742 is not backported). - Ran against Trino master

Re: Location of rust repo

Thank you for the context, Chan. This morning I created a separate repository (iceberg-rust ) for the rust implementation. With PyIceberg we already did separate releases from the Java artifacts, also the versioning is different. I think this is an excellent

Re: Proposal to fix the docs - this time it'll be different

Hey Brian, Thanks for raising this. As a release manager, I can confirm that the current structure is confusing, and I can also see the community struggling with this because they are willing to contribute to the docs, but cannot always find the place where to do this. I think the complexity of th

Re: Discussion about the location of language clients

Hi everyone, Today I took a stab at the generation of wheels in Python (here's the PR if anyone is interested), and when testing this it would also kick off many unrelated CI jobs. This is just for two languages, and I'm not convinced that it will scal

Re: Discussion about the location of language clients

vide proper links to repos for different languages. > > > > I would be -1 on renaming apache/iceberg to apache/iceberg-java, as it can > break external links to the main/original github repo. the tradeoff may not > be worth it. > > > > On Thu, Aug 10, 2023 at 8:16 AM Fo

PyIceberg 0.5.0 release

Hi everyone, As mentioned in the latest Iceberg sync, I'd love to do another Python release. I would like to reach out to the community to see if there is anything we should include. I know that it was promised that the next release would have to write support, but so many features have already a

[VOTE] Release Apache PyIceberg 0.5.0

Hi everyone I propose that we release the following RC as the official PyIceberg 0.5.0 release. The commit ID is 5bd7c649e4743a61eace5f52517db9b5b56ff8e6 * This corresponds to the tag: pyiceberg-0.5.0rc1 ( 4f314fc507dec4ae918d3a3dfba567a28f92ac22) * https://github.com/apache/iceberg/releases/tag

Re: [VOTE] Release Apache PyIceberg 0.5.0

steps > Ran tests via "make test-coverage" > Ran glue integration tests > > Best regards, > Jonas > > On Fri, Sep 8, 2023 at 3:19 PM Hussein Awala wrote: > >> +1 (non binding) I ran the example notebooks and tested some queries >> with PyArrow and Pandas &g

Re: [VOTE] Release Apache PyIceberg 0.5.0

apache/iceberg/issues/8530#issuecomment-1712482320>. Rusty jumped in <https://github.com/apache/iceberg/issues/8530#issuecomment-1712493430> and we both learned that an unsigned long on Windows is 32bits. Once those are in, we're ready for another RC. Cheers, Fokko Op za 9 sep 2023 om 09

[VOTE] Release Apache PyIceberg 0.5.0

the next 72 hours. [ ] +1 Release this as PyIceberg 0.5.0 [ ] +0 [ ] -1 Do not release this because... Please consider this my +1, I've checked against the docker-spark-iceberg <https://github.com/tabular-io/docker-spark-iceberg/pull/92> notebook, and did some checks. Kind regards, Fokko Driesprong

[VOTE] Release Apache PyIceberg 0.5.0 RC3

Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.5.0 release. This includes the performance issue that was discovered in RC2. A summary of what's included in 0.5.0: - Add gzip metadata support - PyArrow HDFS s

Re: [VOTE] Release Apache PyIceberg 0.5.0 RC3

./dev/check-license This will be part of the docs once released. Kind regards, Fokko Op wo 13 sep 2023 om 14:18 schreef Fokko Driesprong : > Hi Everyone, > > > I propose that we release the following RC as the official PyIceberg 0.5.0 > release. This includes the performance

Re: [VOTE] Release Apache PyIceberg 0.5.0 RC3

ture made Wed 13 Sep 2023 08:07:40 PM CST >> gpg: using RSA key FCD3779E399C53D995FC82A35171BA3E54493550 >> gpg:issuer "fo...@apache.org" >> gpg: Good signature from "Fokko Driesprong " [ultimate] >> gpg: Signature made

Re: [VOTE] Release Apache PyIceberg 0.5.0 RC3

I've corrected it for RC3 as well, so we don't need to send out another RC. Cheers, Fokko Op za 16 sep 2023 om 22:32 schreef Fokko Driesprong : > Hey Ryan, > > Thanks for catching that. It slipped in here > <https://github.com/apache/iceberg/commit/bf748dab6cf986f54f42

[ANNOUNCE] PyIceberg 0.5.0

I'm pleased to announce the release of Apache PyIceberg 0.5.0! PyIceberg 0.5.0 comes with many new features: - Add gzip metadata support - PyArrow HDFS support - Support serverless environments (A

Re: [VOTE] Release Apache PyIceberg 0.5.0 RC3

eader looks ok >>> - no binaries found in the pyiceberg distribution which is good >>> >>> Thanks ! >>> Regards >>> JB >>> >>> On Wed, Sep 13, 2023 at 2:18 PM Fokko Driesprong >>> wrote: >>> > >>> > Hi E

Re: [VOTE] Release Apache Iceberg 1.4.0 RC1

+1 (binding) Thanks Anton for running the release and everyone who contributed! Checks I did: - Updated the docker-spark-iceberg repo , and everything runs fine (still with Spark 3.4 since there were some problems with Jupyte

Migration of PyIceberg to iceberg-python repository

ind regards, Fokko Driesprong

Re: Migration of PyIceberg to iceberg-python repository

history. > For example: > - > https://medium.com/@ayushya/move-directory-from-one-repository-to-another-preserving-git-history-d210fa049d4b > > - https://gist.github.com/trongthanh/2779392 > > Please give it a try. > > Thanks, > Ajantha > > On Fri, Sep 29, 202

Re: Migration of PyIceberg to iceberg-python repository

pull requests, but I assume there are very >>>> few. >>>> >>>> On Fri, Sep 29, 2023 at 10:34 AM Pucheng Yang >>>> wrote: >>>> >>>>> Thanks for doing this. I wonder how do we deal with all the issues >>>>> filed

Re: Migration of PyIceberg to iceberg-python repository

from the iceberg repo. Kind regards, Fokko Op za 30 sep 2023 om 21:06 schreef Fokko Driesprong : > Hey everyone, > > Pucheng: I wonder how do we deal with all the issues filed for python >> module but still open in iceberg repo? > > > That's a good point. I think we should

Re: [DISCUSSION] Rename master branch as main for the main repository

Big +1! Thanks for raising this JB! Kind regards, Fokko Op di 3 okt 2023 om 07:56 schreef Jean-Baptiste Onofré : > Thanks all for your feedback. > > I will prepare the renaming then, I will keep you posted. > > Regards > JB > > On Tue, Oct 3, 2023 at 2:36 AM Renjie Liu wrote: > > > > +1 > > >

Re: [PROPOSAL] Regular release pace & some post release actions

My 2ct, There is no harm in stating it explicitly, however, I'm not in favor of making it so explicit by pinning a date onto it (Jan 24). I would rather say that releases can be expected at least every quarter (so it doesn't need to be updated :) I noticed that the releases of Iceberg are also dr

Re: [PROPOSAL] Regular release pace & some post release actions

track the release progress using github milestones > especially as we get closer to the release date, which provides more > context. Tracking in multiple places just leads to inconsistency. > > -Dan > > > > On Sat, Oct 7, 2023 at 11:09 AM Fokko Driesprong wrote: > &g

Re: Migration of PyIceberg to iceberg-python repository

egards, Fokko Op ma 2 okt 2023 om 16:16 schreef Fokko Driesprong : > Hey everyone, > > Update from my side. I've moved all the issues > <https://github.com/apache/iceberg-python/issues> and my PRs > <https://github.com/apache/iceberg-python/pulls>. Not all issues

PyIceberg 0.5.1 patch release

Hey everyone, This week we've discovered a serious bug when parsing SQL-like string expressions (thanks Pucheng for reporting this)! Ryan suggested doing a quick patch release to get this fix out to the public ASAP, and I think that's a great idea

Re: Iceberg Slack invite

Hey Lin, Can you try this link: https://join.slack.com/t/apache-iceberg/shared_invite/zt-2561tq9qr-UtISlHgsdY3Virs3Z2_btQ The link you mentioned is working for me, but I'm already part of the workspace. Can you share the error that you're seeing? Kind regards, Fokko Op ma 16 okt 2023 om 22:39 sc

[VOTE] Release Apache PyIceberg 0.5.1 (RC1)

Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.5.1 release. This is a patch release due to a bug that has been found . Smaller bugs also have been backported

Re: [VOTE] Release Apache PyIceberg 0.5.1 (RC1)

re good > > - source distribution is good > > - run a quick test locally > > > > Thanks, > > Regards > > JB > > > > On Mon, Oct 16, 2023 at 11:28 PM Fokko Driesprong > wrote: > >> > >> Hi Everyone, > >> > >> > >> I p

Re: [VOTE] Release Apache Iceberg 1.4.1 RC0

Thanks Eduard for running this release! +1 (binding): - Checked the sha/signature - Ran our example notebooks against 1.4.1 and it looks well Xuanwo, if you want to learn more about voting, there is also an Apache page on

Re: Request access to iceberg slack channel

Hey Alessio, Everyone is welcome on the Iceberg slack. What kind of error are you seeing? Can you try this link: https://join.slack.com/t/apache-iceberg/shared_invite/zt-2561tq9qr-UtISlHgsdY3Virs3Z2_btQ Kind regards, Fokko Op do 19 okt 2023 om 19:11 schreef Alessio Izzo : > Hello, > I'd like to

Re: [VOTE] Release Apache PyIceberg 0.5.1 (RC1)

en able to verify the set of changes. I was >>>> looking at the 0.5.0-rc3 tag in the main repo: >>>> https://github.com/apache/iceberg/commit/f798b06246e67131d413dfceece5ccaf269e01fe >>>> >>>> I don't see that commit in the 0.5.x branch. Wher

[VOTE] Release Apache PyIceberg 0.5.1 RC2

Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.5.1 release. This is a patch release due to bugs: - Part of the expression is ignored when multiple and/or expressions are specified - Update like statements to

Re: [VOTE] Release Apache Iceberg 1.4.2 RC0

Thanks the quick followup Amogh! +1 (binding) Verified sigs/sums/license/build and ran against our example notebooks . Kind regards, Fokko Op ma 30 okt 2023 om 04:42 schreef Daniel Weeks : > +1 (binding) > > Verified sigs/sums/licen

Re: [VOTE] Release Apache PyIceberg 0.5.1 RC2

Thanks everyone for voting! I'll go ahead with the release since we have 5 positive votes: +1 Rushan (non-binding) +1 JB (non-binding) +1 Dan Weeks (binding) +1 Ryan Blue (binding) +1 Fokko Driesprong (binding) Thanks everyone, and I'll send out the announcement when all the art

[ANNOUNCE] PyIceberg 0.5.1 release

>. It is highly recommended to update to this version. Thanks everyone for contributing! Kind regards, Fokko Driesprong

Re: Updating the Iceberg table architecture diagram

Hey Jason, thanks for updating the chart. I like it a lot. However, there are a lot of boxes and new terms. What do you think of keeping both files, and indicating that the old applies to V1 tables, and the new one to V2 tables. Kind regards, Fokko Op vr 3 nov 2023 om 14:37 schreef Aaron Niskode

Re: Add me to slack channel

Hey Sardar, Please use the following URL: https://join.slack.com/t/apache-iceberg/shared_invite/zt-2561tq9qr-UtISlHgsdY3Virs3Z2_btQ The slack channel should be public, let me know what you ran into. We're always happy to answer questions. Kind regards, Fokko Op zo 5 nov 2023 om 20:02 schreef Sa

Re: Slack community

Hey, Thanks for reaching out. I'll make sure to update the Slack URL. Can you check using: https://join.slack.com/t/apache-iceberg/shared_invite/zt-27f22riz7-o8nCsl5Vbc_2h6~3DF6qlw Kind regards, Fokko O

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

ck and issue pass. I plan to start the > >> release tomorrow. > >> > >> Regards > >> JB > >> > >> On Mon, Nov 20, 2023 at 8:56 AM Driesprong, Fokko > wrote: > >> > > >> > I took the liberty and created a 1.4.3 milestone

Re: Is there a way to distcp iceberg table from hadoop?

Hi Dongjun, Thanks for reaching out on the mailinglist. Another option might be to copy the data, and then use a Spark procedure, called add_files to add the files to the table. Let me know if this works for you. Kind regards, F

Re: Proposal for REST APIs for Iceberg table scans

Hey Rahil and Jack, Thanks for bringing this up. Ryan and I also discussed this briefly in the early days of PyIceberg and it would have helped a lot in the speed of development. We went for the traditional approach because that would also support all the other catalogs, but now that the REST cata

Re: [DISCUSS] Apache Iceberg 1.4.3

+1 Would be great to have the abovementioned fix out to the public, and some other small fixes are worth releasing (see milestone)! Thanks, Fokko Op di 19 dec 2023 om 08:55 schreef Ajantha Bhat : > +1 for 1.4.3 release with #9227 ASAP. > > Looks like Trino does manual retry and this issue is m

Re: [PROPOSAL] Improvement on our PR flows

Nice! I fully agree with the abovementioned. I originally set up the stalebot for the issues because I noticed that there were many issues around old Spark versions that weren't even maintained anymore. I feel it is better to either close or take action on an issue. For me, it makes sense to extend

[ANNOUNCE] New committer: Honah J.

On behalf of the Iceberg PMC, I'm happy to announce that Honah has accepted an invitation to become a committer on Apache (Py)Iceberg. Welcome, and thank you for your contributions! Kind regards, Fokko

Re: Proposed PyIceberg logo art

Love it Rick, thanks for sharing! I would love to have it as the official PyIceberg logo! I've checked the trademark of the Python logo, and they are okay with using the logo for non-proprietary use. They recommend checking it with the PSF anyway since we combine logos here, so I'll do that now. I

Re: Proposed PyIceberg logo art

>> >>> On Tue, Jan 16, 2024 at 7:25 AM Zheng Hu wrote: >>> > >>> > The logo looks great and lovely, thanks Rick. >>> > >>> > But I'm not sure whether we need a separate Python Iceberg logo for >>> the pyiceberg, which

[DISCUSS] PyIceberg 0.6.0 release

ork out to the public so they can try it and we can uncover any impediments as soon as possible. And we can follow up with 0.7.0. Kind regards, Fokko Driesprong

Re: [DISCUSS] PyIceberg 0.6.0 release

t; >>> Really great job to everyone involved. >>> >>> Best, >>> Sung >>> >>> On Jan 26, 2024, at 11:48 AM, Ryan Blue wrote: >>> >>>  >>> It's great to see all the progress in PyIceberg. Thanks to everyone

Re: [DISCUSS] Release new Iceberg docs site in the main repository

I did some reviews of the PRs that led up to this, and I think the new site is much easier to maintain and deploy. +1 from my end :) Cheers, Fokko Op ma 29 jan 2024 om 15:15 schreef Jean-Baptiste Onofré : > +1 > > Regards > JB > > On Fri, Jan 26, 2024 at 11:40 PM Brian Olsen > wrote: > > > > He

Re: [DISCUSS] PyIceberg 0.6.0 release

ith the >> limitations as it's still a huge step forward and we can build >> incrementally on the write support. >> >> Incredible work everyone, I'm really excited about the progress here. >> >> -Dan >> >> On Fri, Jan 26, 2024 at 11:16 AM Fokko D

Re: [DISCUSS] iceberg-rust 0.2.0 release

I'm all for the 0.2.0 release. Kudos to all the work so far. While the functionality is limited today, a lot of things are already in progress and it looks very promising. Also, running a release now will help to streamline the release process. Kind regards, Fokko Driesprong Op wo 31 jan 20

Re: [DISCUSS] Change iceberg-rust CI Settings to only require approval for new github users

much faster. Also from a reviewer perspective, I like to know if the CI passes before reviewing and this also takes a bit of time. I don't think there is much risk since the Actions have limited permissions, and all the repositories are actively looked at. Kind regards, Fokko Driesprong Op wo 31 j

Re: [PROPOSAL] Create user mailing list ?

±0 for having a user mailing list. I don't believe that having more channels will lead to better support. I agree that the archiving capabilities of Slack are limited, and the search is sub-optimal. But we should also make sure that the questions asked are also integrated into the documentation. T

Re: [Discuss] Change iceberg-python and iceberg-go CI Settings to only require approval for first time contributors

+1 Op vr 2 feb 2024 om 08:47 schreef Eduard Tudenhoefner : > +1 > > > > On Fri 2. Feb 2024 at 04:56 Drew wrote: > >> +1 >> >> Thanks for bringing this up for PyIceberg Honah >> >> On Thu, Feb 1, 2024 at 5:35 PM Honah J. wrote: >> >>> Hello everyone >>> >>> Inspired by our recent discussion rega

Re: [DISCUSS] iceberg-rust 0.2.0 release

ss here. >> >> -Dan >> >> On Wed, Jan 31, 2024 at 9:36 AM Fokko Driesprong >> wrote: >> >>> I'm all for the 0.2.0 release. Kudos to all the work so far. While the >>> functionality is limited today, a lot of things are already in prog

Re: [VOTE] Release Apache PyIceberg 0.6.0rc4

Hi Justin, Dan, Thanks for checking this. For the Avro one, we copied parts of the decompression and binary decoder for the internal PyIceberg implementation (that reads from an Iceberg schema, rather than from an Avro schema). I checked the Avro NOTICE, and there isn't anything relevant. I notic

Re: [DISCUSS] iceberg-rust 0.2.0 release

Hey Renjie, That would be great. I'm happy to do the committer/PMC side of things. Let's coordinate on the release tracking issue: https://github.com/apache/iceberg-rust/issues/180 Kind regards, Fokko Kind regards, Fokko Driesprong Op wo 7 feb 2024 om 03:36 schreef Xuanwo : &g

Re: [VOTE] Release Apache PyIceberg 0.6.0rc4

That makes sense. I've updated the PR: https://github.com/apache/iceberg-python/pull/410/ PTAL. Kind regards, Fokko Op zo 11 feb 2024 om 03:58 schreef Justin Mclean : > HI, > > For the Thrift and Hive ones, we have an optional dependency that ships > the content under the vendor/ directory: > ht

[VOTE] Release Apache Iceberg Rust 0.2.0 RC1

Hello, Apache Iceberg Rust Community, This is a call for a vote to release Apache Iceberg Rust version 0.2.0. The tag to be voted on is 0.2.0-rc.1. This first release provides integration with the REST catalog and a lot of scaffolding that's needed for reading the data. The release candidate:

Java Iceberg 2.0: Hadoop upgrade

Hi everyone, I want to discuss adding the Hadoop upgrade to the list after moving to Iceberg 2.0. We still compile against Hadoop 2.7.3 to ensure we support as many users as possible. Hadoop 2.7.3 was released August 2016 and is not maintained anymore

Re: [VOTE] Release Apache PyIceberg 0.6.0rc6

+1 (binding) I've checked signatures and checksums, checked the licenses, and did some checks around writing. Kind regards, Fokko Op ma 19 feb 2024 om 03:07 schreef Amogh Jahagirdar : > +1 non-binding > Verified signatures, checksum, and license > Ran unit/integ tests on Python 3.10.4 > Ran ad-

Re: [VOTE] Release Apache Iceberg Rust 0.2.0 RC1

error: no file found at: /home/blue/tmp/apache-iceberg-rust-0.2.0-src >>make: *** [Makefile:33: cargo-sort] Error 1 >> >> >> >> On Mon, Feb 19, 2024 at 11:00 AM Jack Ye wrote: >> >>> +1 (binding) >>> >>> Verified checksum, signatur

Re: [VOTE] Release Apache Iceberg 1.5.0 RC0

Just using this thread to come back to the NOTICE discussion. This came also up with the latest Python release, and I spent quite a bit of time on it. If it's "used" section is not strictly required in NOTICE from a legal > perspective, the embedded dependencies should be mentioned (either > under

Re: Gravitino an Iceberg REST catalog service

Hey everyone, Thanks for raising this. I think a test-jar would be a great first step. We already maintain "service" considering JDBC, Hive, etc catalogs. REST Catalog ref impl in Iceberg would be the sam. What I think Ryan means by a service is having to maintain Postgres (JDBC backend), Hive

Re: [VOTE] Release Apache Iceberg 1.5.0 RC4

+1 (binding) - Checked checksum and signature - Ran a modified version of dbt-spark to take advantage of the views, and it worked like a charm! 🥳 Cheers, Fokko Op vr 1 mrt 2024 om 06:43 schreef Ajantha Bhat : > Gentle reminder. > > On Wed, Feb 28, 2024 at 8:34 PM Eduard Tudenhoefner > wrote: >

New committer: Bryan Keller

Hi everyone, The Project Management Committee (PMC) for Apache Iceberg has invited Bryan Keller to become a committer and we are pleased to announce that he has accepted. Bryan was contributing to Iceberg before it was even open-source, did a lot of work on the topic of metadata generation, and i

Re: [VOTE] Release Apache Iceberg 1.5.0 RC6

+1 (binding) Thanks again for working on this Ajantha and Eduard. - Checked checksum and signature - Ran a modified version of dbt-spark to take advantage of the views and it worked great! Cheers, Fokko Op za 9 mrt 2024 om 06:35 schreef Szehon Ho : > +1 (binding) > > * Verified signature > *

New committer: Renjie Liu

Hi everyone, The Project Management Committee (PMC) for Apache Iceberg has invited Renjie Liu to become a committer and we are pleased to announce that he has accepted. We're very excited to have Renjie as a committer as he's leading the effort of bringing Iceberg to the Rust world. Being a commi

Re: [ANNOUNCE] Apache Iceberg release 1.5.0

Thanks for running the release Ajantha. It is great to see view support being released on the Java side 🎉 Thanks everyone for the hard work in making this release happen! Including all our new contributors ! Kind regards, Fokko

Re: [DISCUSS] Iceberg board report - March 2024

Thanks Ryan, That looks comprehensive, thanks for taking the time to compile the report. I have a few suggestions for the release section: - Name the releases by name: Python → PyIceberg. If people want to look it up, just googling the name will bring them to it directly. - Split the rel

Re: [DISCUSS] What do we plan for Iceberg 2.0.0 ?

Hey JB, Thanks for raising this. Sorry for the late reply, but I was OOO last week. I think in general the progress is being kept on the spec itself . Also, some features are already available (default values in Python, and nanosecond timestamps

Re: [PROPOSAL] Improvement on our PR flows

this? Kind regards, Fokko Driesprong Op wo 13 mrt 2024 om 13:17 schreef Renjie Liu : > Hi, JB: > > Your proposal looks great to me. We should definitely have a vote for a > proposal impacting the spec, and the model is great. > > On Tue, Mar 12, 2024 at 10:55 PM Jean-Baptiste Ono

Re: [VOTE] Release Apache PyIceberg 0.6.1rc1

+1 (binding) - Checked the signature and the checksum - Ran the example notebooks against 0.6.1rc1 - Did some checks locally and looks all good! Thanks Honah for running the release! Kind regards, Fokko Op do 4 apr 2024 om 17:56 schr

Re: [VOTE] Release Apache PyIceberg 0.6.1rc1

Hey everyone, First of all thanks for all the votes. Regarding the discussion around the NOTICE. We all agree that when something is bundled, it needs to be added to the notice. However, Laynes Law of Debate comes into play: what's the definition of bundling? To e

Re: Looking for help with Pyflink and Iceberg

Hey Frank, Thanks for reaching out here. I spent some cycles a while ago to remove the Hadoop requirement from Flink. There were a lot of APIs that needed to change, which caused not to follow through with it. But this might help you in getting PyFlink up and running since it contains an example s

Re: [VOTE] Release Apache PyIceberg 0.6.1rc2

Hey everyone, First of all, thanks Honah for running the release! +1 (binding) from my end - I checked the signature, hashes, and licenses and all look good . - Ran some local tests. Kind regards, Fokko Op di 16 apr 2024 om 05:55

Re: [VOTE] Release Apache PyIceberg 0.6.1rc2

re is the poetry.lock file that provides reproducable CI builds, and this is missing from the tar.gz (where it will try to install the latest and greatest). Kind regards, Fokko Driesprong Op do 18 apr 2024 om 04:21 schreef Kevin Liu : > +1 (non binding) > > Downloaded specific commit f

Re: [VOTE] Release Apache PyIceberg 0.6.1rc3

Thanks Honah for the quick follow-up with RC3. +1 binding - Ran the signatures, checksums, and licenses. - Double-checked that it installs from a clean Python 3.10 doc

Re: [VOTE] Release Apache Iceberg 1.5.1 RC0

Sorry for being late to the party! +1 (binding) - Checked checksum, signature and licenses - Ran example notebooks Kind regards, Fokko Op di 23 apr 2024 om 22:58 schre

Re: [ANNOUNCE] Apache PyIceberg release 0.6.1

Awesome! Thanks for running this release Honah 🙌 Kind regards, Fokko Op wo 1 mei 2024 om 06:48 schreef Honah J. : > I'm pleased to announce the release of Apache PyIceberg 0.6.1! > > Apache Iceberg is an open table format for huge analytic datasets. Iceberg > delivers high query performance for

Re: [VOTE] Release Apache Iceberg 1.5.2 RC0

+1 (binding) Thanks for going through this once more! - Ran the signatures and checksums - Checked the licenses - Ran some sample checks with Spark 3.5 (Scala 2.12) Kind regards, Fokko Op do 2 mei 2024 om 15:51 schreef Eduard Tudenhoefner : > +1 (non-binding) > > * validated checksum and signa

Re: GitHub issue labels

Hey Manu, I don't explicitly use the labels, but they help me to categorize the issues mentally. I agree that there is room for improvement as there are more issues being raised every day. Other communities also have interesting approaches, such as: - Triage label: When a new bug, improvement

Re: Addressing security questions in the Iceberg REST specification

Hey Robert, Sorry for the late reply as I was out last week. I'm not an OAuth guru either, but some context from my end. * Credentials (for example username/password) must _never_ be sent to > the resource server, only to the authorization server. In an earlier discussion

Re: Addressing security questions in the Iceberg REST specification

T servers to opt for integrating with any standard >>> OAuth2 / >>> >>>> OIDC provider (e.g. Okta, Keycloak, Authelia). >>> >>>> >>> >>>> I agree with both of these points; again I don't think the >>> intentio

Re: [INFO] Preparing the Apache Iceberg 1.6.0 release

Hi JB, thanks for raising this. - With the Gradle version update, we will be able to upgrade to Parquet > 1.14.0 We might want to defer this until Parquet 1.14.1 gets released. There is an issue found with Jackson that prohibits Spark from upgrad

Re: Agenda Community Sync 19th June

Hey Jan, Thanks for raising this. Let me jot down the highlights, and feel free to add what you'd like to discuss. I'm personally looking forward to an update on the materialized views. Kind regards, Fokko Op di 18 jun 2024 om 20:28 schreef Jan Kaul : > Hi all, > > I was wondering whether there

Re: Agenda Community Sync 19th June

Hey everyone, Thanks for the input. I've collected everything in the notes , feel free to do suggestions or edits. Thanks Brian for running the recording. Looking forward to seeing everyone later today! Kind regards

  1   2   3   >