Re: [VOTE] Release Apache PyIceberg 0.1.0 RC2

2022-09-29 Thread Ye, Jack
+1 Ran checksum, checked license and signature, ran unit tests. Ran against Hive catalog and S3 with CLI, tested create/load/drop/rename table and create/drop/load database. Best, Jack Ye From: Daniel Weeks Reply-To: "dev@iceberg.apache.org" Date: Wednesday, September 28, 2022 at 9:37 PM To:

Re: [DISCUSS] Semantic Versioning Guarantees post 1.0

2022-09-29 Thread Ye, Jack
+1 for the proposed policy, and the list looks good to me as well. The minor version deprecations required category is indeed what most end application users I see depend on directly. Best, Jack Ye From: Daniel Weeks Reply-To: "dev@iceberg.apache.org" Date: Wednesday, September 28, 2022 at 4:

Re: Iceberg tables in AWS Redshift

2022-09-29 Thread Ye, Jack
Hi, We have this PR that could generate symlink manifests to be used in Redshift as of today, this is a similar experience as Delta Lake integration. I know multiple organizations that are doing this: https://github.com/apache/iceberg/pull/5398. For plans of native integration offering, please

Re: [DISCUSS] Semantic Versioning Guarantees post 1.0

2022-09-29 Thread Ryan Blue
+1 to the approach outlined here. Thanks, Dan! On Wed, Sep 28, 2022 at 4:18 PM Daniel Weeks wrote: > Hey Iceberg Community, > > I wanted to raise a discussion thread with respect to how to handle > semantic versioning and deprecations so that we can document the > expectations for changes to the

Reverting a commit in the table history?

2022-09-29 Thread Pucheng Yang
Hi all, I wonder if any discussion happened about the idea of reverting a commit in the table history? My clients have such a use case: they are writing some data into a partition, and later want to revert that. But since there are new snapshots generated, thus they can not use snapshot rollback.

Re: Reverting a commit in the table history?

2022-09-29 Thread Ye, Jack
Hi, There is a PR published just today for something similar that you might be able to reference: https://github.com/apache/iceberg/pull/5888, which rolls back a compaction commit on conflict and then reapply the changes. The logic seems to be similar as what you want, to rollback to that speci

Re: [DISCUSS] Semantic Versioning Guarantees post 1.0

2022-09-29 Thread Yufei Gu
+1 for the proposed approach. Here is a question and a minor suggestion. Question: Do we consider the module iceberg-data as the minor version compatible guaranteed? Suggestion: I get that we don’t want modules like spark/flink to be minor and major versions compatible. It’s simple, and works wel

Re: Reverting a commit in the table history?

2022-09-29 Thread Pucheng Yang
Thank you, I will take a look. On Thu, Sep 29, 2022 at 2:40 PM Ye, Jack wrote: > Hi, > > > > There is a PR published just today for something similar that you might be > able to reference: https://github.com/apache/iceberg/pull/5888, which > rolls back a compaction commit on conflict and then re

Re: [VOTE] Release Apache PyIceberg 0.1.0 RC2

2022-09-29 Thread Anton Okolnychyi
+1 - Anton > On Sep 29, 2022, at 9:42 AM, Ye, Jack wrote: > > +1 > > Ran checksum, checked license and signature, ran unit tests. > Ran against Hive catalog and S3 with CLI, tested create/load/drop/rename > table and create/drop/load database. > > Best, > Jack Ye > > From: Daniel Weeks