Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

2022-10-10 Thread Ye, Jack
+1 (binding) Checked license, signature Ran checksum Ran build and test in java 8 and 11 Ran against EMR 6.8 Spark and Trino Best, Jack Ye From: Daniel Weeks Reply-To: "dev@iceberg.apache.org" Date: Monday, October 10, 2022 at 4:12 PM To: "dev@iceberg.apache.org" Subject: RE: [VOTE] Release A

Re: Iceberg Contribution

2022-10-04 Thread Ye, Jack
Hi, Welcome to the community! We have the contribution guideline at https://iceberg.apache.org/contribute/. Looks like the page is lacking some information about how people can get started. Typically you can find issues labelled as “good first issue” at https://github.com/apache/iceberg/labels

Re: Proposal - Priority based commit ordering on partitions

2022-10-03 Thread Ye, Jack
Friday 10/7 after 1:30pm pacific time also works for me. But I know Prashant you are at Indian timezone, we should probably see what time works the best from that perspective. Best, Jack Ye From: Ryan Blue Reply-To: "dev@iceberg.apache.org" Date: Friday, September 30, 2022 at 8:53 AM To: "dev

Re: Reverting a commit in the table history?

2022-09-29 Thread Ye, Jack
Hi, There is a PR published just today for something similar that you might be able to reference: https://github.com/apache/iceberg/pull/5888, which rolls back a compaction commit on conflict and then reapply the changes. The logic seems to be similar as what you want, to rollback to that speci

Re: Iceberg tables in AWS Redshift

2022-09-29 Thread Ye, Jack
Hi, We have this PR that could generate symlink manifests to be used in Redshift as of today, this is a similar experience as Delta Lake integration. I know multiple organizations that are doing this: https://github.com/apache/iceberg/pull/5398. For plans of native integration offering, please

Re: [DISCUSS] Semantic Versioning Guarantees post 1.0

2022-09-29 Thread Ye, Jack
+1 for the proposed policy, and the list looks good to me as well. The minor version deprecations required category is indeed what most end application users I see depend on directly. Best, Jack Ye From: Daniel Weeks Reply-To: "dev@iceberg.apache.org" Date: Wednesday, September 28, 2022 at 4:

Re: [VOTE] Release Apache PyIceberg 0.1.0 RC2

2022-09-29 Thread Ye, Jack
+1 Ran checksum, checked license and signature, ran unit tests. Ran against Hive catalog and S3 with CLI, tested create/load/drop/rename table and create/drop/load database. Best, Jack Ye From: Daniel Weeks Reply-To: "dev@iceberg.apache.org" Date: Wednesday, September 28, 2022 at 9:37 PM To:

AWS DynamoDB Catalog Support

2021-06-08 Thread Ye, Jack
Hi everyone, Because of the many asks we have received for supporting AWS DynamoDB Catalog implementation, I have put up PR https://github.com/apache/iceberg/pull/2688 showing the DynamoDbCatalog that we designed and implemented. I have heard a few internal implementations based on DynamoDB in

Re: Key rotation in Iceberg data encryption

2021-03-25 Thread Ye, Jack
Yes, I totally agree with Russell that key rotation should be treated as something like a rewrite manifest action, and when the rewrite completes, the old files with old keys can be expired in a separated snapshot expiration action. Because of requirements like GDPR, this expiration would happen

Re: Extending Apache Iceberg Encryption Module

2021-03-24 Thread Ye, Jack
Sounds good, lets continue with some discussions through the doc. For POC, I think what you conclude is mostly correct, I am currently implementing the encryption spec, general encrypted file stream with KMS API, and I would expect the low level file encryption integration to take place separate

Re: Adobe Blog ..

2021-01-16 Thread Ye, Jack
Nice! +1 for a blog page, AWS will also post a few blogs after 0.11 that can be added later. From: Carl Steinbach Reply-To: "dev@iceberg.apache.org" Date: Friday, January 15, 2021 at 16:54 To: Iceberg Dev List , Ryan Blue Cc: Russell Spitzer , "ssad...@adobe.com" , Filip Bocse , Xabriel Colla

Iceberg 0.11.0 Release Plan

2021-01-13 Thread Ye, Jack
Hi everyone, This is Jack Ye from AWS, and I will be the release manager for Iceberg 0.11.0. The purpose of this email is to start the preparation of this release. Overview We have discussed with groups of people working on different areas of the codebase, and the current plan is to have the i