+1 (non-binding)
Verified all steps.
Just a bit of heads up in case anyone encounters the same issue, when
running unit tests on my laptop, my local custom HADOOP_HOME and
HADOOP_CONF_DIR settings somehow caused 30 iceberg-flink unit tests to fail
and "TestFlinkIcebergSink" to hang. After unsetting these 2 env variables,
all unit tests passed. I don't think it should block the RC.
On Thu, Nov 5, 2020 at 8:50 AM Daniel Weeks <dwe...@apache.org> wrote:
> +1 (binding)
>
> Verified all steps.
>
> On Thu, Nov 5, 2020 at 8:31 AM Anton Okolnychyi
> <aokolnyc...@apple.com.invalid> wrote:
>
>> +1 (binding)
>>
>> Verified the RC locally.
>>
>> - Anton
>>
>> On 5 Nov 2020, at 08:15, Edgar Rodriguez <
>> edgar.rodrig...@airbnb.com.INVALID> wrote:
>>
>> +1 non-binding for RC4. Tested with internal tests in cluster, validated
>> Spark write and Hive reads.
>>
>> On Thu, Nov 5, 2020 at 5:56 AM Mass Dosage <massdos...@gmail.com> wrote:
>>
>>> +1 non-binding on RC4. I tested out the Hive read path on a distributed
>>> cluster using HadoopTables.
>>>
>>> On Thu, 5 Nov 2020 at 04:46, Dongjoon Hyun <dongjoon.h...@gmail.com>
>>> wrote:
>>>
>>>> +1 for 0.10.0 RC4.
>>>>
>>>> Bests,
>>>> Dongjoon.
>>>>
>>>> On Wed, Nov 4, 2020 at 7:17 PM Jingsong Li <jingsongl...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> 1. Download the source tarball, signature (.asc), and checksum
>>>>> (.sha512): OK
>>>>> 2. Import gpg keys: download KEYS and run gpg --import
>>>>> /path/to/downloaded/KEYS (optional if this hasn’t changed) : OK
>>>>> 3. Verify the signature by running: gpg --verify
>>>>> apache-iceberg-xx.tar.gz.asc: OK
>>>>> 4. Verify the checksum by running: sha512sum -c
>>>>> apache-iceberg-xx.tar.gz.sha512 : OK
>>>>> 5. Untar the archive and go into the source directory: tar xzf
>>>>> apache-iceberg-xx.tar.gz && cd apache-iceberg-xx: OK
>>>>> 6. Run RAT checks to validate license headers: dev/check-license: OK
>>>>> 7. Build and test the project: ./gradlew build (use Java 8) : OK
>>>>>
>>>>> Best,
>>>>> Jingsong
>>>>>
>>>>> On Thu, Nov 5, 2020 at 7:38 AM Ryan Blue <rb...@netflix.com.invalid>
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> - Validated checksum and signature
>>>>>> - Ran license checks
>>>>>> - Built and ran tests
>>>>>> - Queried a Hadoop FS table created with 0.9.0 in Spark 3.0.1
>>>>>> - Created a Hive table from Spark 3.0.1
>>>>>> - Tested metadata tables from Spark
>>>>>> - Tested Hive and Hadoop table reads in Hive 2.3.7
>>>>>>
>>>>>> I was able to read both Hadoop and Hive tables created in Spark from
>>>>>> Hive using:
>>>>>>
>>>>>> add jar /home/blue/Downloads/iceberg-hive-runtime-0.10.0.jar;
>>>>>> create external table hadoop_table
>>>>>> stored by 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
>>>>>> location 'file:/home/blue/tmp/hadoop-warehouse/default/test';
>>>>>> select * from hadoop_table;
>>>>>>
>>>>>> set iceberg.mr.catalog=hive;
>>>>>> select * from hive_table;
>>>>>>
>>>>>> The hive_table needed engine.hive.enabled=true set in table
>>>>>> properties by Spark using:
>>>>>>
>>>>>> alter table hive_table set tblproperties ('engine.hive.enabled'='true')
>>>>>>
>>>>>> Hive couldn’t read the #snapshots metadata table for Hadoop. It
>>>>>> failed with this error:
>>>>>>
>>>>>> Failed with exception
>>>>>> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>> java.lang.ClassCastException: java.lang.Long cannot be cast to
>>>>>> java.time.OffsetDateTime
>>>>>>
>>>>>> I also couldn’t read the Hadoop table once iceberg.mr.catalog was
>>>>>> set in my environment, so I think we have a bit more work to do to clean
>>>>>> up
>>>>>> Hive table configuration.
>>>>>>
>>>>>> On Wed, Nov 4, 2020 at 12:54 AM Ryan Murray <rym...@dremio.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 (non-binding)
>>>>>>>
>>>>>>> 1. Download the source tarball, signature (.asc), and checksum
>>>>>>> (.sha512): OK
>>>>>>> 2. Import gpg keys: download KEYS and run gpg --import
>>>>>>> /path/to/downloaded/KEYS (optional if this hasn’t changed) : OK
>>>>>>> 3. Verify the signature by running: gpg --verify
>>>>>>> apache-iceberg-xx.tar.gz.asc: I got a warning "gpg: WARNING: This key
>>>>>>> is
>>>>>>> not certified with a trusted signature! gpg: There is no
>>>>>>> indication that the signature belongs to the owner." but it passed
>>>>>>> 4. Verify the checksum by running: sha512sum -c
>>>>>>> apache-iceberg-xx.tar.gz.sha512 : OK
>>>>>>> 5. Untar the archive and go into the source directory: tar xzf
>>>>>>> apache-iceberg-xx.tar.gz && cd apache-iceberg-xx: OK
>>>>>>> 6. Run RAT checks to validate license headers: dev/check-license: OK
>>>>>>> 7. Build and test the project: ./gradlew build (use Java 8 & Java
>>>>>>> 11) : OK
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Nov 4, 2020 at 2:56 AM OpenInx <open...@gmail.com> wrote:
>>>>>>>
>>>>>>>> +1 for 0.10.0 RC4
>>>>>>>>
>>>>>>>> 1. Download the source tarball, signature (.asc), and checksum
>>>>>>>> (.sha512): OK
>>>>>>>> 2. Import gpg keys: download KEYS and run gpg --import
>>>>>>>> /path/to/downloaded/KEYS (optional if this hasn’t changed) : OK
>>>>>>>> 3. Verify the signature by running: gpg --verify
>>>>>>>> apache-iceberg-xx.tar.gz.asc: OK
>>>>>>>> 4. Verify the checksum by running: sha512sum -c
>>>>>>>> apache-iceberg-xx.tar.gz.sha512 : OK
>>>>>>>> 5. Untar the archive and go into the source directory: tar xzf
>>>>>>>> apache-iceberg-xx.tar.gz && cd apache-iceberg-xx: OK
>>>>>>>> 6. Run RAT checks to validate license headers: dev/check-license: OK
>>>>>>>> 7. Build and test the project: ./gradlew build (use Java 8) : OK
>>>>>>>>
>>>>>>>> On Wed, Nov 4, 2020 at 8:25 AM Anton Okolnychyi <
>>>>>>>> aokolnyc...@apple.com.invalid> wrote:
>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>> I propose the following RC to be released as official Apache
>>>>>>>>> Iceberg 0.10.0 release.
>>>>>>>>>
>>>>>>>>> The commit id is d39fad00b7dded98121368309f381473ec21e85f
>>>>>>>>> * This corresponds to the tag: apache-iceberg-0.10.0-rc4
>>>>>>>>> *
>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-0.10.0-rc4
>>>>>>>>> *
>>>>>>>>> https://github.com/apache/iceberg/tree/d39fad00b7dded98121368309f381473ec21e85f
>>>>>>>>>
>>>>>>>>> The release tarball, signature, and checksums are here:
>>>>>>>>> *
>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-0.10.0-rc4/
>>>>>>>>>
>>>>>>>>> You can find the KEYS file here (make sure to import the new key
>>>>>>>>> that was used to sign the release):
>>>>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>>>>
>>>>>>>>> Convenience binary artifacts are staged in Nexus. The Maven
>>>>>>>>> repository URL is:
>>>>>>>>> *
>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1012
>>>>>>>>>
>>>>>>>>> This release includes important changes:
>>>>>>>>>
>>>>>>>>> * Flink support
>>>>>>>>> * Hive read support
>>>>>>>>> * ORC support fixes and improvements
>>>>>>>>> * Application of row-level delete files on read
>>>>>>>>> * Snapshot partition summary
>>>>>>>>> * Ability to load LocationProvider dynamically
>>>>>>>>> * Sort spec
>>>>>>>>>
>>>>>>>>> Please download, verify, and test.
>>>>>>>>>
>>>>>>>>> Please vote in the next 72 hours.
>>>>>>>>>
>>>>>>>>> [ ] +1 Release this as Apache Iceberg 0.10.0
>>>>>>>>> [ ] +0
>>>>>>>>> [ ] -1 Do not release this because…
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Anton
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ryan Blue
>>>>>> Software Engineer
>>>>>> Netflix
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best, Jingsong Lee
>>>>>
>>>>
>>
>> --
>> Edgar R
>>
>>
>>
--
John Zhuge
