Re: [VOTE] Release Apache PyIceberg 0.9.1rc1

2025-04-26 Thread Sung Yun
Thank you Fokko for running this release! +1 (non-binding) I verified signatures, checksums, the LICENSE and NOTICE. Tested with python 3.11. Sung On Sat, Apr 26, 2025 at 10:51 AM Manu Zhang wrote: > +1 (non-binding) > > Built and tested with python 3.10. > > Thanks, > Manu > > On Sat, Apr 2

Re: [DISCUSS] Table Identifiers in Iceberg View Spec

2025-04-16 Thread Sung Yun
Thank you Walaa for the proposal. I think view portability is a very important topic for us to continue discussing as it relies on many assumptions within the data ecosystem for it to function like you've highlighted well in the document. I've added a few comments around how this may impact the pe

[ANNOUNCE] Apache Iceberg Pre-Summit Community Meetup in SF

2025-03-11 Thread Sung Yun
Best Regards, Sung Yun

Re: [VOTE] Release Apache PyIceberg 0.9.0rc3

2025-03-03 Thread Sung Yun
Hi Drew, thank you very much for running the release! +1 non-binding: I've verified the signatures, the checksums and the licensing. Ran tests with "make test-coverage" Sung On 2025/03/03 05:01:18 "Honah J." wrote: > Thanks Drew for running the release! > > +1 (binding) > >- Checked signa

Re: [DISCUSS] Rest Catalog 419 Response Code

2025-02-22 Thread Sung Yun
t;>> Thanks, >>> Dmitri. >>> >>> On Thu, Feb 20, 2025 at 7:01 PM Daniel Weeks wrote: >>>> >>>> Hey Sung, >>>> >>>> My interpretation is that it's up to the REST Server to decide whether to >>>> send a

[DISCUSS] Rest Catalog 419 Response Code

2025-02-19 Thread Sung Yun
Hi folks, While working with the Polaris community on an issue[1], I decided to bring this discussion to the Iceberg community mailing list as I believe that the status code 419 in the REST Catalog Open API Spec may have become a source of confusion for the community. In the Iceberg REST Catalog

Re: Welcome Huaxin Gao as a committer!

2025-02-06 Thread Sung Yun
That's fantastic news Huaxin. Congratulations! On 2025/02/06 13:40:09 Rodrigo Meneses wrote: > Congrats and best wishes !!! > > On Thu, Feb 6, 2025 at 5:04 AM Gidon Gershinsky wrote: > > > Congrats Huaxin! > > > > Cheers, Gidon > > > > > > On Thu, Feb 6, 2025 at 2:46 PM Tushar Choudhary < > > t

Re: [VOTE] Deprecate IRC snapshot-id Field of SetStatisticsUpdate

2025-01-21 Thread Sung Yun
+1 non-binding Thanks for driving this Christian! On 2025/01/21 12:39:26 Russell Spitzer wrote: > +1 > > On Tue, Jan 21, 2025 at 4:34 AM Alex Dutra > wrote: > > > +1 (nb) > > > > On Tue, Jan 21, 2025 at 11:30 AM Piotr Findeisen < > > piotr.findei...@gmail.com> wrote: > > > >> +1 non-binding >

Re: [ANNOUNCE] Release Apache Iceberg Rust v0.4.0

2024-12-24 Thread Sung Yun
t 1:31 PM Fokko Driesprong wrote: > > > >> Thanks for driving this release, Sung, and thanks to everyone who > >> contributed! > >> > >> Kind regards, > >> Fokko > >> > >> Op di 24 dec 2024 om 05:22 schreef Xuanwo : > >> &g

[ANNOUNCE] Release Apache Iceberg Rust v0.4.0

2024-12-23 Thread Sung Yun
Hi all, The Apache Iceberg Rust community is pleased to announce that Apache Iceberg Rust v0.4.0 has been released! The notable changes since v0.3.0 include: - Caching parsed Manifests and ManifestLists for performance - Support projection pushdown in Datafusion Iceberg - Utilize row group and ro

Re: [VOTE] Release Apache Iceberg Rust 0.4.0 RC3

2024-12-23 Thread Sung Yun
t;> x86_64-unknown-linux-gnu, x86_64-unknown-linux-musl, > >> aarch64-unknown-linux-musl > >> and aarch64-apple-darwin. > >> > >> > >> On 23. Dec 2024, at 17:45, Barron Wei wrote: > >> > >> +1 non-binding > >> > >>

RE: [VOTE] Release Apache Iceberg Rust 0.4.0 RC3

2024-12-23 Thread Sung Yun
valid > [x] Checksums and signatures > [x] LICENSE/NOTICE files exist > [x] No unexpected binary files > [x] All source files have ASF headers > [x] Can compile from source > Ran verify.py and make test on macOS M2 with OrbStack. > On 2024/12/20 15:13:01 Sung Yun wrote: >

[VOTE] Release Apache Iceberg Rust 0.4.0 RC3

2024-12-20 Thread Sung Yun
Hello, Apache Iceberg Rust Community, This is a call for a vote to release Apache Iceberg rust version 0.4.0. The tag to be voted on is v0.4.0-rc.3. This RC includes the following fixes: - Reading a table with positional deletes should fail - verify.py script should be executable The release ca

Re: [VOTE] Release Apache Iceberg Rust 0.4.0 RC2

2024-12-19 Thread Sung Yun
o binary file found in the source distribution > > - build is ok (I just had to fix an local issue with cargo but it was > > on my machine) > > > > Regards > > JB > > > > On Wed, Dec 18, 2024 at 2:22 AM Sung Yun wrote: > > > > > > Hello, Apac

[VOTE] Release Apache Iceberg Rust 0.4.0 RC2

2024-12-17 Thread Sung Yun
Hello, Apache Iceberg Rust Community, This is a call for a vote to release Apache Iceberg rust version v0.4.0-rc.2. The tag to be voted on is v0.4.0-rc.2. The release candidate: https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-rust-0.4.0-rc.2/ Keys to verify the release candidate:

Re: [VOTE] Release Apache Iceberg Rust 0.4.0 RC1

2024-12-17 Thread Sung Yun
release and reporting the issue! Sung On Tue, Dec 17, 2024 at 5:43 PM Sung Yun wrote: > > Hi Kevin, > > Thanks for the speedy response, and for reporting the issue! > > It sounds like the wrong credentials may have been used for signing. And my > self-verifying the signatur

Re: [VOTE] Release Apache Iceberg Rust 0.4.0 RC1

2024-12-17 Thread Sung Yun
.0-src.tar.gz: OK > ``` > > The verify script requires `chmod` to execute, but this is not a blocker. > ``` > chmod +x ./scripts/verify.py > ``` > > Best, > Kevin Liu > > On Tue, Dec 17, 2024 at 1:50 PM Sung Yun wrote: > >> Hello, Apache Iceberg Rust Commun

[VOTE] Release Apache Iceberg Rust 0.4.0 RC1

2024-12-17 Thread Sung Yun
Hello, Apache Iceberg Rust Community, This is a call for a vote to release Apache Iceberg rust version v0.4.0-rc.1. The tag to be voted on is v0.4.0-rc.1. The release candidate: https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-rust-0.4.0-rc.1/ Keys to verify the release candidate:

Re: New committer: Matt Topol

2024-12-10 Thread Sung Yun
Congratulations Matt! On 2024/12/10 12:49:25 Alex Dutra wrote: > Congratulations, Matt! Go!! > > On Tue, Dec 10, 2024 at 1:08 PM Péter Váry > wrote: > > > Congratulations Matt! > > > > On Tue, Dec 10, 2024, 12:58 Ian Cook wrote: > > > >> Congratulations Matt!!! > >> > >> On Tue, Dec 10, 2024 a

Re: [VOTE] Release Apache PyIceberg 0.8.1rc1

2024-12-03 Thread Sung Yun
> >> become the default for local execution.We recommend switching to the > >> NativeRunner now via `daft.context.set_runner_native()` or by setting the > >> env variable `DAFT_RUNNER=native`. Please report any issues at > >> github.com/Eventual-Inc/Daft/issues >

Re: [DISCUSS] iceberg rust 0.4.0 and iceberg pyiceberg_core 0.1.0 release

2024-11-28 Thread Sung Yun
://rust.iceberg.apache.org/release.html) for Iceberg-Rust, I've started a tracking issue to help work towards the 0.4.0 release: https://github.com/apache/iceberg-rust/issues/739 > I am willing to implement the release scripts needed for Sung Yun to execute > the release. And yes, I'd g

Re: [VOTE] Release Apache PyIceberg 0.8.1rc1

2024-11-27 Thread Sung Yun
prevent this from happening again. > https://github.com/apache/iceberg-python/pull/1386 > > Best, > Kevin Liu > > On Wed, Nov 27, 2024 at 5:07 PM Sung Yun wrote: > >> Hi Kevin, >> >> Thank you so much for working on this release! >> >> I noticed this

Re: [VOTE] Release Apache PyIceberg 0.8.1rc1

2024-11-27 Thread Sung Yun
Hi Kevin, Thank you so much for working on this release! I noticed this morning that PyIceberg 0.8.1 was released and yanked[1] this morning. Similar to how we had handled it when this had happened last time, I think this would mean that we would need to now move on to the next version and publis

Re: [DISCUSS] iceberg rust 0.4.0 and iceberg pyiceberg_core 0.1.0 release

2024-11-27 Thread Sung Yun
re as well. > > > > Two cents about the iceberg-rust release schedule: it seems too aggressive to > release by 2 weeks, monthly(4 weeks) release would be a nice fit. > > Sent from my iPhone > > > > > On Sep 5, 2024, at 8:25 PM, Sung Yun wrote

Re: [ACTION REQUIRED] Removal of v3 artifact actions on December 5th

2024-11-27 Thread Sung Yun
Hi JB and Kevin, thank you for jumping on the chore. Here's one more PR to bump up the version in iceberg-rust: https://github.com/apache/iceberg-rust/pull/725 I assume this didn't show up in the grep.app search since it was recently merged On 2024/11/26 22:22:36 Kevin Liu wrote: > We merged th

Re: [DISCUSS] Apache Iceberg Summit 2025 - Selection Committee

2024-11-26 Thread Sung Yun
Hi JB, thank you very much for starting this thread! I'd love to volunteer as well. Sung On 2024/11/26 18:22:13 Yufei Gu wrote: > Super excited about the summit. I’d like to help with that. > > Yufei > > > On Tue, Nov 26, 2024 at 10:16 AM Kevin Liu wrote: > > > Very excited about this. Happ

Re: [VOTE] Release Apache PyIceberg 0.8.0rc2

2024-11-15 Thread Sung Yun
Hi Kevin, Thank you again for running this release! I've verified the License headers, checksums and signatures. Downloaded the RC from SVN and ran the tests. Downloaded the package from pypi and ran sanity checks. +1 (non-binding) Sung On 2024/11/14 20:56:44 Kevin Liu wrote: > Hi Everyone,

Re: [PROPOSAL] Create Iceberg DockerHub repository

2024-11-14 Thread Sung Yun
Hi JB, That sounds great!! The REST TCK /adapter docker image will be super useful for the Iceberg subprojects as it will ensure that they have access to a light-weight REST Catalog Server image with the latest features to run integration tests against. Sung On 2024/11/14 15:41:04 Jean-Baptis

Re: [VOTE] Release Apache PyIceberg 0.8.0rc1

2024-11-13 Thread Sung Yun
Hi folks, While testing out the Rest Catalog Adapter docker image that Ajantha has been working working on, I ran into an issue when parsing the TableResponse of a staged table. While the metadata-location is an optional field according to the Iceberg Rest Catalog Spec, the field is being hand

Re: [VOTE] Release Apache PyIceberg 0.8.0rc1

2024-11-07 Thread Sung Yun
Hi Kevin, Thank you for running this release! It's a +1 from me. I've installed pyiceberg==0.8.0rc1 from the Apache SVN and verified the checksums, the signature, and the license headers. I've also checked that the installation works from pip as well. Sung On 2024/11/07 21:57:51 Kevin Liu wr

Re: [DISCUSS] Discrepancy Between Iceberg Spec and Java Implementation for Snapshot summary's 'operation' key

2024-10-19 Thread Sung Yun
points about being able to remove old table versions. I don't > think that the REST protocol itself is the place to do this. The protocol > is format-agnostic. Implementations are free to reject requests to create > tables with older versions, or to update the table to a new version. > &

Re: [DISCUSS] Discrepancy Between Iceberg Spec and Java Implementation for Snapshot summary's 'operation' key

2024-10-18 Thread Sung Yun
t; > On Thu, Oct 17, 2024 at 1:56 PM Anton Okolnychyi > > wrote: > > > >> Well, the spec says nothing about a top-level `operation` field in JSON > >> [1]. Yet the Java implementation produces it [2] and removes the operation > >> from the summary map. This seems

Re: [DISCUSS] Discrepancy Between Iceberg Spec and Java Implementation for Snapshot summary's 'operation' key

2024-10-17 Thread Sung Yun
91", > >> "added-data-files" : "231513", > >> "manifests-replaced" : "0", > >> "total-data-files" : "231513" > >> } > >> ``` > >> > >> It could be t

Re: [DISCUSS] Discrepancy Between Iceberg Spec and Java Implementation for Snapshot summary's 'operation' key

2024-10-17 Thread Sung Yun
Thank you for the clarification Daniel, and thank you Kevin for raising this issue! Does that mean that we are creating component schemas that are the superset of the V1 and V2 schemas? And if so, should we remove summary and manifest-list from the required properties, and add manifests optiona

Re: [DISCUSS] [PyIceberg] Use of asserts to "programming the negative space"

2024-10-12 Thread Sung Yun
Hi André, Thank you for starting off this discussion! This is a fun topic, so I’m keen on seeing what the rest of the folks in the PyIceberg community think as well :) I’m of the opinion that ‘assert’ should only be used within test suites, because setting the optimize flag (-O) in the Python int

Re: [DISCUSS] Iceberg Rust Sync Meeting

2024-10-11 Thread Sung Yun
Thank you for starting this thread Xuanwo, I'm +1 for a Iceberg Rust meeting. Regarding the meeting time, I believe the Iceberg Catalog Community sync happens two consecutive weeks, at the same time as the Iceberg community sync, when there isn't the tri-weekly Iceberg meeting. For example, if

Re: [DISCUSS] Iceberg Summit 2025 ?

2024-09-30 Thread Sung Yun
Hi JB, thank you for starting this thread! I’m already very excited at the prospect of connecting with other members of the community. I think it would be nice to organize one next year given the success of the 2024 Summit. > should we have another event ? Yes! Definitely > would you like there

Re: [Notice] Update to catalog sync meeting timezone 2

2024-09-24 Thread Sung Yun
Thank you Kevin! Sung On 2024/09/24 17:51:54 Kevin Liu wrote: > https://docs.google.com/document/d/1iPGVCIcr-M0XtAiudOguWAvmqIdVgpYN5vz5ohO8PKw/edit > This doc includes the calendar for the catalog sync and notes from past > syncs. > > Best, > Kevin Liu > > On Tue, Sep

Re: [Notice] Update to catalog sync meeting timezone 2

2024-09-24 Thread Sung Yun
Hi Jack Ye, thank you for the update ! This may be a silly question, but where can I find the Catalog sync calendar and it's meeting details? Does it share the same google meets link as the Iceberg community sync? Sung On 2024/09/24 15:11:10 Jack Ye wrote: > Hi everyone, > > Due to the low at

Re: [VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-24 Thread Sung Yun
Thank you everyone for participating in the vote to drop support for Python3.8 The vote result is: +1: 4 (binding), 2 (non-binding) - Kevin Liu (non-binding) - Yufei Gu (binding) - Steven Wu (binding) - Ryan Blue (binding) - Ameena Ansari (non-binding) - Daniel Weeks (binding) +0: 0 (binding), 0

[VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-20 Thread Sung Yun
Hi folks, I'd like to start this thread to vote on dropping the support for Python3.8 in the upcoming 0.8.0 PyIceberg release. Python3.8 will be End-Of-Life in October 2024, and some of our dependencies have already dropped support for Python3.8 prebuilt wheels which makes our dependency manageme

Re: [DISCUSS] iceberg rust 0.4.0 and iceberg pyiceberg_core 0.1.0 release

2024-09-05 Thread Sung Yun
Thank you for driving this Xuanwo! +1 as well, as noted the 0.1.0 pyiceberg_core release will allow PyIceberg to begin integrating with the rust based core and introduce a new feature that the community is looking for. On Thu, Sep 5, 2024 at 6:05 AM Renjie Liu wrote: > +1 for this release. > >

[DISCUSS] PyIceberg 1.0.0 Release

2024-08-29 Thread Sung Yun
Hi folks, With the recent releases, the interest in PyIceberg has been growing at a very exciting speed: https://pypistats.org/packages/pyiceberg. And this got me thinking about the roadmap for a 1.0.0 release. Looking up for guidelines for "when to make a 1.0.0 release", the answer on semver.org

[DISCUSS] iceberg-rust: pyiceberg_core 0.1.0 Release

2024-08-27 Thread Sung Yun
Hi folks, We’ve recently had a few discussion threads on components we wanted to expose as Python Bindings from the iceberg-rust project in the form of the "pyiceberg_core" package. Now that the Python bindings for the Transforms have been implemented, I wanted to open this thread to discuss the

Re: [VOTE] REST Endpoint discovery

2024-08-20 Thread Sung Yun
+1 (non-binding) Thank you Eduard! This is a great feature enhancement to the catalog. Thumbs up! On Tue, Aug 20, 2024 at 5:27 PM Amogh Jahagirdar <2am...@gmail.com> wrote: > +1 > Thanks for driving this Eduard! > > On Tue, Aug 20, 2024 at 3:08 PM Ryan Blue > wrote: > >> +1 >> >> On Tue, Aug 20

Re: [DISCUSS] Use iceberg-rust for PyIceberg Bucket Transform

2024-08-18 Thread Sung Yun
read initiated by xuanwo. I think these are all >>> intermediate steps to our final goal. >>> >>> >>> On Fri, Aug 2, 2024 at 3:21 AM Ryan Blue >>> wrote: >>> >>>> In general, I think the idea of using iceberg-rust more from PyIceberg &

[ANNOUNCE] Apache PyIceberg release 0.7.1

2024-08-18 Thread Sung Yun
Hi everyone, I'm pleased to announce the release of Apache PyIceberg 0.7.1! Again, this patch release includes the following important bug fixes: * Fix `delete` to trace existing manifests when a data file is partially rewritten * Fix 'to_arrow

Re: [VOTE] Release Apache PyIceberg 0.7.1rc2

2024-08-17 Thread Sung Yun
unning the release! Thanks everyone > for contributing and testing! > > Best regards, > Honah > > On Fri, Aug 16, 2024 at 5:04 PM Sung Yun wrote: > >> Hi folks! >> >> We are 1 binding vote short of accepting this release candidate. The >> verificatio

Re: [VOTE] Release Apache PyIceberg 0.7.1rc2

2024-08-16 Thread Sung Yun
with the comments that this doesn't require a new RC. >> >> +1 (binding) >> >> Verified sigs/sums/license/build/test with Python 3.11.9 >> >> Thanks, >> -Dan >> >> On Thu, Aug 15, 2024 at 3:34 PM Sung Yun wrote: >> >>> Hi D

Re: [VOTE] Release Apache PyIceberg 0.7.1rc2

2024-08-15 Thread Sung Yun
>>> Kind regards, >>> Fokko >>> >>> Op wo 14 aug 2024 om 19:57 schreef André Luis Anastácio >>> : >>> >>>> >>>>- validated signatures and checksums >>>> >>>> >>>>- checked licens

[VOTE] Release Apache PyIceberg 0.7.1rc2

2024-08-13 Thread Sung Yun
Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.7.1 release. A summary of the high level features: * Fix `delete` to trace existing manifests when a data file is partially rewritten * Fix 'to_arrow_batch_rea

Re: Welcome Péter, Amogh and Eduard to the Apache Iceberg PMC

2024-08-13 Thread Sung Yun
Congratulations Péter, Amogh and Eduard! On 2024/08/13 20:25:17 Russell Spitzer wrote: > Hi Y'all, > > It is my pleasure to let everyone know that the Iceberg PMC has voted to > have several talented individuals join us. > > So without further ado, please welcome Péter Váry, Amogh Jahagirdar an

Re: [VOTE] Release Apache PyIceberg 0.7.1rc1

2024-08-12 Thread Sung Yun
ay > > On Sat, Aug 10, 2024 at 2:15 AM André Luis Anastácio > wrote: > > > > >- validated signatures and checksums > > > > > >- checked license > > > > > >- ran tests and test-coverage with Python 3.9.12 > > > > >

[VOTE] Release Apache PyIceberg 0.7.1rc1

2024-08-09 Thread Sung Yun
Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.7.1 release. This is a patch release due to the following bugs: * Fix correctness of applying positional deletes on Merge-On-Read tables * Fix overwrite when f

Re: [DISCUSS] PyIceberg 0.7.1 release

2024-08-09 Thread Sung Yun
Hey Sung, > > That's a great find. I just merged the PR, and it would be good to get the > release process rolling to get #1026 > <https://github.com/apache/iceberg-python/pull/1026> out to the users. > > Kind regards, > Fokko > > Op do 8 aug 2024 om 23:20 schre

Re: [DISCUSS] PyIceberg 0.7.1 release

2024-08-08 Thread Sung Yun
p di 6 aug 2024 om 21:17 schreef André Luis Anastácio > : > >> What do you think about adding the fix that excludes PyIceberg support >> for Python 3.9.7 in the 0.7.1 release?[1] It already doesn't work, so this >> is just to avoid any new issues. >> >> - [1]: http

Re: [DISCUSS] PyIceberg: Remove optional support for instance-level identifier in Catalog and Table APIs

2024-08-06 Thread Sung Yun
we can completely remove the catalog name. > > > > Thanks, > > Kevin Liu > > On Wed, Jul 31, 2024 at 2:56 PM Sung Yun wrote: > >> Today in PyIceberg, we have support for identifier parsing in public APIs >> belonging to two different classes: >> >&

Re: [DISCUSS] PyIceberg 0.7.1 release

2024-08-06 Thread Sung Yun
our > users. > > +1 on this concern. Is it possible to make the Arrow 17.0.0 upgrade > optional first? So that folks who want the upgrade can test it out. > > Thanks, > Kevin Liu > > > > On Fri, Aug 2, 2024 at 11:33 AM Sung Yun wrote: > > > Hi Fokk

Re: [DISCUSS] PyIceberg 0.7.1 release

2024-08-02 Thread Sung Yun
berg-python/pull/929>, so we can push down > the large/small type to PyArrow (only for to_arrow), and apply #986 > <https://github.com/apache/iceberg-python/pull/986> on top if you want to > force it to either small or large types. > > WDYT? > > Kind regards, > Fokk

Re: [DISCUSS] PyIceberg 0.7.1 release

2024-08-02 Thread Sung Yun
still open for any new issues that are identified from 0.7.0 release, that we should fix in the patch release. Thank you, Sung On 2024/07/30 23:57:04 Sung Yun wrote: > Hi folks, > > We are starting to compile the list of issues to fix and port into the > 0.7.1 release. > > Th

[DISCUSS] Use iceberg-rust for PyIceberg Bucket Transform

2024-08-01 Thread Sung Yun
Hi everyone, This is something I've been mulling about for a while and I thought this would be the right forum to discuss this topic as a follow up to a similar topic discussion thread on using python bindings from iceberg-rust to support pyiceberg. As soon as we released 0.7.0 which supports wri

[DISCUSS] PyIceberg: Remove optional support for instance-level identifier in Catalog and Table APIs

2024-07-31 Thread Sung Yun
Today in PyIceberg, we have support for identifier parsing in public APIs belonging to two different classes: - Catalog class: load_table, purge_table, drop_table - Table class: scan These APIs currently have optional support for the identifier that the instance itself belongs to. For ex

[DISCUSS] PyIceberg 0.7.1 release

2024-07-30 Thread Sung Yun
Hi folks, We are starting to compile the list of issues to fix and port into the 0.7.1 release. The current list of known issues is as follows: Fix pydantic warning on table commit: #972 (thanks for the quick fix ndrluis!) Issue when rewriting

[ANNOUNCE] Apache PyIceberg release 0.7.0

2024-07-30 Thread Sung Yun
I'm pleased to announce the release of Apache PyIceberg 0.7.0! Once again, this large release includes the following features on a high level: * Write support to partitioned tables with IdentityTransform and TimeTransform partitions * Support for deletes using predicates. It will drop whole files

Re: [VOTE] Release Apache PyIceberg 0.7.0rc2

2024-07-30 Thread Sung Yun
;>>>> kevin.jq@gmail.com> wrote: >>>>> >>>>> +1 (non-binding) >>>>> Verified signatures/checksums/license. Ran unit and integration tests. >>>>> Logs are attached to this email. >>>>> >>>>> Siden

Re: [VOTE] Release Apache PyIceberg 0.7.0rc2

2024-07-30 Thread Sung Yun
-binding) >>> - validated signatures & checksums >>> - checked license - RAT checks passed >>> - ran tests and test-coverage with Python 3.9 >>> >>> Thank you everyone for the hard work! >>> >>> Best, >>> Chinmay >>&g

Re: Meeting time for catalog community sync

2024-07-28 Thread Sung Yun
Hi Renjie, I found that the recordings were on the official Iceberg YouTube channel under the Iceberg Community Syncs playlist . Sung On Sun, Jul 28, 2024 at 5:57 AM Renjie Liu wrote: > Hi: > > This may

Re: [VOTE] Release Apache PyIceberg 0.7.0rc2

2024-07-27 Thread Sung Yun
;2. If there are things found that need fixing, we can bump the version >to 0.7.1 > > Wish you all a great weekend, > > Kind regards, > Fokko > > Kind regards, > Fokko > > Op za 27 jul 2024 om 03:45 schreef Sung Yun : > >> Hi ndrluis, >> >

Re: [VOTE] Release Apache PyIceberg 0.7.0rc2

2024-07-26 Thread Sung Yun
PyPi before starting a new vote. Sung On Fri, Jul 26, 2024 at 9:02 PM wrote: > Hey Sung Yun, > > Thank you for starting the release. > > I was checking PyPI, and it looks like the release candidate was published > as version 0.7.0 (https://pypi.org/project/pyiceberg/0.7.0/).

[VOTE] Release Apache PyIceberg 0.7.0rc2

2024-07-26 Thread Sung Yun
Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.7.0 release. This is a large release featuring many amazing contributions from the community, and here’s a summary of the features introduced on a high level: * Write support to partitioned tables with Identit

Re: [ANNOUNCE] Welcoming new committers and PMC members

2024-07-23 Thread Sung Yun
:03, Fokko Driesprong wrote:Hi everyone,The Iceberg PMC is excited to announce new committers and PMC members to the Apache Iceberg project.New committers:Kevin Liu (kevinjqliu)Piotr Findeisen (findepi)Sung Yun (syun64)Xuanwo (xuanwo)New members of the PMC:Honah (honahx)Renjie Liu (liurenjie1024)We’re ve

Re:Table Schema History Pruning

2024-02-20 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi Barron, we've noticed the same issue as well since this PR was merged in to introduce schema versions: https://github.com/apache/iceberg/pull/2096 There's a closed issue where folks were discussions options in remediating this problem, that also has links to other related PRs and Issues: htt

[ANNOUNCE] Apache PyIceberg release 0.6.0

2024-02-19 Thread Sung Yun (BLOOMBERG/ 120 PARK)
I'm pleased to announce the release of Apache PyIceberg 0.6.0! Apache Iceberg is an open table format for huge analytic datasets. Iceberg delivers high query performance for tables with tens of petabytes of data, along with atomic commits, concurrent writes, and SQL-compatible table evolution. Th

Re: [VOTE] Release Apache PyIceberg 0.6.0rc6

2024-02-19 Thread Sung Yun (BLOOMBERG/ 120 PARK)
>>>>>> >>>>>> Ryan >>>>>> >>>>>> On Thu, Feb 15, 2024 at 7:55 AM Uwe L. Korn wrote: >>>>>> >>>>>>> Hello all, >>>>>>> >>>>>>> just wanted to gi

[VOTE] Release Apache PyIceberg 0.6.0rc6

2024-02-14 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi Everyone, We are moving onto the next RC with some important fixes. This RC includes: * Bug Fix in passing configuration through environment variables #423 * Arm wheels #416 * Correction to the NOTICE and LICENSE #413 Again, here's a summary of the high level features included in this release

[VOTE] Release Apache PyIceberg 0.6.0rc5

2024-02-12 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi everyone, Apologies for the many RC's, but as people are actively trying out PyIceberg and write support, we continue to find bugs and improvements. This RC includes: - Fixed missing locking mechanism for the Hive commit path #405 - Correctly produce V1 metadata for non-REST catalogs #411 I

Re: [VOTE] Release Apache PyIceberg 0.6.0rc4

2024-02-10 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi everyone, thank you very much for testing out the RC and submitting your votes We would like to unfortunately cancel the current RC and send out a new one that will include the following two critical fixes: - Hive locking - Issue with creating V1 tables with Hive, SQL and Glue catalogs. Tha

[VOTE] Release Apache PyIceberg 0.6.0rc4

2024-02-08 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi Everyone, Apologies for the many RC's, but as people are actively trying out PyIceberg and write support, we continue to find bugs and improvements. This RC includes: - Support converting Arrow int8, int16 and large-string types. - Fixed a discrepancy between the Daft docs and code. - Properl

[VOTE] Release Apache PyIceberg 0.6.0rc3

2024-02-06 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi Everyone, We've identified a few more issues in PyIceberg 0.6.0rc2, and hence we will be cancelling that RC, and proceed on to the next. I propose that we release the following RC as the official PyIceberg 0.6.0 release. This is the third RC that includes the following fixes: * Handle nul

[VOTE] Release Apache PyIceberg 0.6.0rc2

2024-02-05 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.6.0 release. This is the second RC that patches: - Parquet compression is not being set (#345) - Remove the constraint on sort-orders (#349) - Issues when writing null-columns (#337, #348) Again, here's a summa

Re: [VOTE] Release Apache PyIceberg 0.6.0rc1

2024-02-02 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Jan 31, 2024, at 8:33 AM, Pucheng Yang wrote: nvm, I was under the wrong impression it was released already. Thanks. On Wed, Jan 31, 2024 at 8:31 AM Pucheng Yang wrote: 0.6.0 has been released already. Do you mean to release 0.6.1? On Wed, Jan 31, 2024 at 8:26 AM Sung Yun (BLOOMBERG/ 120 PARK

[VOTE] Release Apache PyIceberg 0.6.0rc1

2024-01-31 Thread Sung Yun (BLOOMBERG/ 120 PARK)
Hi Everyone, I propose that we release the following RC as the official PyIceberg 0.6.0 release. A summary of the high level features: * Write support for writing to unpartitioned tables * Includes snapshot generation * Constructing Avro writer trees * Support writing metadata which allows to c

Subscribe

2024-01-26 Thread Sung Yun (BLOOMBERG/ 120 PARK)

Re: [DISCUSS] PyIceberg 0.6.0 release

2024-01-26 Thread Sung Yun
It’s so exciting to see the project take another step forward, Fokko!Really great job to everyone involved.Best,SungOn Jan 26, 2024, at 11:48 AM, Ryan Blue wrote:It's great to see all the progress in PyIceberg. Thanks to everyone that's been contributing!I'm all for getting a release out as soon

RE: [DISCUSS] Iceberg community summit

2024-01-18 Thread Sung Yun
:) Sung Yun On 2024/01/12 17:47:08 Ryan Blue wrote: > Hi everyone, > > We've been having discussions about how to put together an Iceberg > conference or summit for this year and one of the first steps is to put > together a selection committee that will be responsible for