Re: New Versioned Iceberg Documentation Site

2022-01-31 Thread Ryan Blue
+1 from me. I think the new site looks great and it is a big improvement to have version-specific docs. Thanks for all your work on this, Sam! On Mon, Jan 31, 2022 at 5:48 PM Sam Redai wrote: > Hey Everyone, > > With 0.13.0's approval for release, I think this would be a good time to > have a di

New Versioned Iceberg Documentation Site

2022-01-31 Thread Sam Redai
Hey Everyone, With 0.13.0's approval for release, I think this would be a good time to have a discussion around the proposed versioned documentation site, powered by Hugo. The site is ready to be released and the source code for the site can be found in the apache/iceberg-docs repository: https://

Re: Continuing the Secondary Index Discussion

2022-01-31 Thread Jack Ye
Hi Zaicheng, I cannot see your pictures, maybe we could discuss in Slack. The goal here is to have a monotonically increasing number that could be used to detect what files have been newly added and should be indexed. This is especially important to know how up-to-date an index is for each partiti

Re: Hive table compatibility for Iceberg readers

2022-01-31 Thread Owen O'Malley
On Thu, Jan 27, 2022 at 10:26 PM Walaa Eldin Moustafa wrote: > *2. Iceberg schema lower casing:* Before Iceberg, when users read Hive > tables from Spark, the returned schema is lowercase since Hive stores all > metadata in lowercase mode. If users move to Iceberg, such readers could > break once

Re: Hive table compatibility for Iceberg readers

2022-01-31 Thread Walaa Eldin Moustafa
Hi everyone, bumping up this thread. Does it help to discuss each feature individually? I am guessing (4. Default value support) is straightforward since it is on the roadmap already. If not, let us start with this. As for the next step, it would be great if we can focus on (1. Reading tables wit

Re: [VOTE] Release Apache Iceberg 0.13.0 RC2

2022-01-31 Thread Jack Ye
Thanks to everyone who participated in the vote for Release Apache Iceberg 0.13.0 RC2. The vote result is: +1: 4 (binding), 7 (non-binding) +0: 0 (binding), 0 (non-binding) -1: 0 (binding), 0 (non-binding) Therefore, the release candidate is passed. Cheers, Jack Ye On Mon, Jan 31, 2022 at 2:07

Re: [VOTE] Release Apache Iceberg 0.13.0 RC2

2022-01-31 Thread Jack Ye
+1 (binding) * checked signature, checksum * Ran unit test, AWS integration test * Tested basic table operations and RewriteDataFiles on AWS EMR Spark 3.0 and 3.1 environments Best, Jack Ye On Mon, Jan 31, 2022 at 1:52 PM Ryan Blue wrote: > +1 (binding) > > * Checked signature, checksums, ran

Re: [VOTE] Release Apache Iceberg 0.13.0 RC2

2022-01-31 Thread Ryan Blue
+1 (binding) * Checked signature, checksums, ran RAT * Ran tests * Tested Spark 3.2 CREATE, INSERT, and INSERT OVERWRITE (same table) * Tested merge-on-read with v1 table * Tested update to v2 * Tested DELETE FROM with merge-on-read * Tested rewrite_data_files to merge deletes * Tested adding a pa

Re: [VOTE] Release Apache Iceberg 0.13.0 RC2

2022-01-31 Thread Armin Najafi
+1 non-binding All tests passed Signature OK Checksum OK License OK Cheers, Armin On Mon, Jan 31, 2022 at 12:54 PM Yufei Gu wrote: > +1 non-binding > > 1. Verify the signature, checksum > 2. All tests passed. > > Best, > > Yufei > > > On Mon, Jan 31, 2022 at 9:45 AM Daniel Weeks wrote: > >>

Re: [VOTE] Release Apache Iceberg 0.13.0 RC2

2022-01-31 Thread Yufei Gu
+1 non-binding 1. Verify the signature, checksum 2. All tests passed. Best, Yufei On Mon, Jan 31, 2022 at 9:45 AM Daniel Weeks wrote: > +1 (binding) > > Verified sigs/sums/license/build/tests. > > I ran some basic tests using spark 3.2 with the new runtime and everything > worked as expected

Re: [VOTE] Release Apache Iceberg 0.13.0 RC2

2022-01-31 Thread Daniel Weeks
+1 (binding) Verified sigs/sums/license/build/tests. I ran some basic tests using spark 3.2 with the new runtime and everything worked as expected. -Dan On Sun, Jan 30, 2022 at 5:53 PM Kyle Bendickson wrote: > +1 (non-binding) > > Verified signature, checksum, rat check, build and ran tests,

Re: [VOTE] Release Apache Iceberg 0.13.0 RC2

2022-01-31 Thread Ashish Singh
+1 (non-binding) - Verified signature, checksum and rat check. - Built and ran tests for all spark versions. - Built spark-3.2 runtime jar and tested with spark-3.2. Tests included. - basic smoke tests on create table, insert and drop table. - custom location provider -