+1, Verified locally following instructions
From: Ryan Blue <b...@apache.org>
Reply-To: "dev@iceberg.apache.org" <dev@iceberg.apache.org>
Date: Tuesday, April 28, 2020 at 5:37 PM
To: Ryan Blue <b...@apache.org>
Cc: Iceberg Dev List <dev@iceberg.apache.org>
Subject: Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
+1 (binding)
Validated checksums and signature, ran ./gradlew check, and ran license check.
I also downloaded Spark 2.4.5 and loaded iceberg-spark-runtime like this:
spark-2.4.5-bin-hadoop2.7/bin/spark-shell \
--conf
spark.jars.repositories=https://repository.apache.org/content/repositories/orgapacheiceberg-1005/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapacheiceberg-1005%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357275264&sdata=%2F2j0GQdcJkJenSuCEFDdxNGCCOESL%2B8R8mjXsp0JO%2F8%3D&reserved=0>
\
--packages org.apache.iceberg:iceberg-spark-runtime:0.8.0-incubating \
--conf spark.sql.warehouse.dir=/home/blue/tmp/spark-warehouse \
--conf spark.hadoop.hive.metastore.uris=thrift://localhost:45655
I spot-checked Hive writes, overwrites, metadata tables, and input_file_name().
By the way, to run a metastore, I just started one of the Iceberg Hive
metastore tests with an infinite loop and added a print statement to get the
thrift URI. That’s an easy way to create one.
On Tue, Apr 28, 2020 at 1:44 PM Ryan Blue
<b...@apache.org<mailto:b...@apache.org>> wrote:
Here are the steps to verify the release that I sent out last time, for anyone
that doesn’t want to look them up:
1. Download the source tarball, signature (.asc), and checksum (.sha512)
from
https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Ficeberg%2Fapache-iceberg-0.8.0-incubating-rc1%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357275264&sdata=XUVa4AXB4sZ5Q0DHURfbZ2a4RoLlTBAUE7YBe3y6uAg%3D&reserved=0>
2. Import gpg keys: download KEYS and run gpg --import
/path/to/downloaded/KEYS (optional if this hasn’t changed)
3. Verify the signature by running: gpg --verify
apache-iceberg-0.8.0-incubating.tar.gz.asc
4. Verify the checksum by running: sha512sum -c
apache-iceberg-0.8.0-incubating.tar.gz.sha512
5. Untar the archive and go into the source directory: tar xzf
apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
6. Run RAT checks to validate license headers: dev/check-license
7. Build and test the project: ./gradlew build (use Java 8)
You can also validate the LICENSE and NOTICE documentation, which is included
in the source tarball, as well as the staged binary artifacts. The latest
update to the spark-runtime Jar’s was PR
#966<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-iceberg%2Fpull%2F966&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357285257&sdata=FOGI9awfJsk2d71Nd1Rbk3imEwOS7vbTI7lEDWvk2QY%3D&reserved=0>
if you’d like to review it.
To validate the convenience binaries, add the Maven URL from the email above to
a downstream project and update your Iceberg dependency to 0.8.0-incubating,
like this:
repositories {
maven {
name 'stagedIceberg'
url
'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapacheiceberg-1005%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357285257&sdata=8u5SE9a%2Fl2yPNxxxT7%2FiwX2aqPz%2BmFAvVp1Yj54hssA%3D&reserved=0>'
}
}
ext {
icebergVersion = '0.8.0-incubating'
}
Then run the downstream project’s tests.
Thanks for reviewing and voting, everyone!
rb
On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue
<b...@apache.org<mailto:b...@apache.org>> wrote:
Hi everyone,
I propose the following RC to be released as official Apache Iceberg
0.8.0-incubating release.
The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
* This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
*
https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-iceberg%2Fcommits%2Fapache-iceberg-0.8.0-incubating-rc1&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357295250&sdata=WuS1FljHsB%2BezfOKy%2F1i7L3Gc1LxBpIw6aPMjSBmisg%3D&reserved=0>
*
https://github.com/apache/incubator-iceberg/tree/4c2dd0ac<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-iceberg%2Ftree%2F4c2dd0ac&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357295250&sdata=wb7sm%2FZ4xGkFjrgZZ0Eo5E8p03iuETWSSdzITnuLelk%3D&reserved=0>
The release tarball, signature, and checksums are here:
*
https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Ficeberg%2Fapache-iceberg-0.8.0-incubating-rc1%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357295250&sdata=otK4CxUcuFgeV9Bj5Kxc7Z2w3ey6ytKX9QRJyzbVT5A%3D&reserved=0>
You can find the KEYS file here:
*
https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Ficeberg%2FKEYS&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357305243&sdata=DbQHT2qQzxgLS9Sa9Yerfmw3oILHL0gAk%2FhHrwdx%2FC0%3D&reserved=0>
Convenience binary artifacts are staged in Nexus. The Maven repository URL is:
*
https://repository.apache.org/content/repositories/orgapacheiceberg-1005/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapacheiceberg-1005%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357305243&sdata=%2FyeX2KAvoam4BQosIhFJQLSzBvSzj1a3N3eIkdQoigw%3D&reserved=0>
This release contains many bug fixes and several new features:
* Actions to remove orphaned files and to optimize metadata for query
performance
* Support for ORC data files
* Snapshot cherry-picking
* Incremental scan planning based on table history
* In and notIn expressions
* An InputFormat for writing MR jobs
Please download, verify, and test.
Please vote in the next 72 hours.
[ ] +1 Release this as Apache Iceberg 0.8.0-incubating
[ ] +0
[ ] -1 Do not release this because...
--
Ryan Blue
--
Ryan Blue
--
Ryan Blue
