There is one issue we may want to fix before the release. We should suppress exceptions while unlocking in HiveTableOperations. Right now, we throw a runtime exception. This leads to cases when the commit is actually successful and we swap the pointer but the subsequent exception during unlock triggers the clean-up logic and we remove valid metadata files. This corrupts the table metadata as HMS will have references to files that are not there. If this happens, people will have to drop the table, find the correct metadata file and recreate the table from it.
I am preparing a fix right now. I’d rather include it in the release. What does everybody think? - Anton > On 29 Apr 2020, at 09:23, RD <rdsr...@gmail.com> wrote: > > +1 . Verified the outlined steps. > > -R > > On Wed, Apr 29, 2020 at 8:09 AM tison <wander4...@gmail.com > <mailto:wander4...@gmail.com>> wrote: > Hi Junjie, > > It is kind of GPG scope issue that you should manually trust Ryan's KEYS. > > FYI https://www.gnupg.org/gph/en/manual/x334.html > <https://www.gnupg.org/gph/en/manual/x334.html> > > Best, > tison. > > > Junjie Chen <chenjunjied...@gmail.com <mailto:chenjunjied...@gmail.com>> > 于2020年4月29日周三 下午9:47写道： > Ran 1-7 steps > found a warning: > gpg: assuming signed data in 'apache-iceberg-0.8.0-incubating.tar.gz' > gpg: Signature made Wed Apr 29 04:07:57 2020 CST > gpg: using RSA key D21CFB9BDBC379681261C7A086781D4FA4B2E9B5 > gpg: Good signature from "Ryan Blue (CODE SIGNING KEY) <b...@apache.org > <mailto:b...@apache.org>>" [unknown] > gpg: aka "Ryan Blue <b...@apache.org > <mailto:b...@apache.org>>" [unknown] > gpg: WARNING: This key is not certified with a trusted signature! > gpg: There is no indication that the signature belongs to the owner. > > +1 (non-binding), if the warning is ok. > > > > > On Wed, Apr 29, 2020 at 7:37 PM Mass Dosage <massdos...@gmail.com > <mailto:massdos...@gmail.com>> wrote: > +1 (non-binding) [I assume only Apache/Iceberg members have binding votes?) > > Similar to others I verified: > > √ RAT checks passed > √ signature is correct > √ checksum is correct > √ build from source > √ run tests locally > > Thanks, > > Adrian > > On Tue, 28 Apr 2020 at 21:45, Ryan Blue <b...@apache.org > <mailto:b...@apache.org>> wrote: > Here are the steps to verify the release that I sent out last time, for > anyone that doesn’t want to look them up: > > Download the source tarball, signature (.asc), and checksum (.sha512) from > https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/ > > <https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/> > Import gpg keys: download KEYS and run gpg --import /path/to/downloaded/KEYS > (optional if this hasn’t changed) > Verify the signature by running: gpg --verify > apache-iceberg-0.8.0-incubating.tar.gz.asc > Verify the checksum by running: sha512sum -c > apache-iceberg-0.8.0-incubating.tar.gz.sha512 > Untar the archive and go into the source directory: tar xzf > apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating > Run RAT checks to validate license headers: dev/check-license > Build and test the project: ./gradlew build (use Java 8) > You can also validate the LICENSE and NOTICE documentation, which is included > in the source tarball, as well as the staged binary artifacts. The latest > update to the spark-runtime Jar’s was PR #966 > <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to > review it. > > To validate the convenience binaries, add the Maven URL from the email above > to a downstream project and update your Iceberg dependency to > 0.8.0-incubating, like this: > > repositories { > maven { > name 'stagedIceberg' > url > 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/ > <https://repository.apache.org/content/repositories/orgapacheiceberg-1005/>' > } > } > > ext { > icebergVersion = '0.8.0-incubating' > } > Then run the downstream project’s tests. > > Thanks for reviewing and voting, everyone! > > rb > > > On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <b...@apache.org > <mailto:b...@apache.org>> wrote: > Hi everyone, > > I propose the following RC to be released as official Apache Iceberg > 0.8.0-incubating release. > > The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392 > * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1 > * > https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1 > > <https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1> > * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac > <https://github.com/apache/incubator-iceberg/tree/4c2dd0ac> > > The release tarball, signature, and checksums are here: > * > https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/ > > <https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/> > > You can find the KEYS file here: > * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS > <https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS> > > Convenience binary artifacts are staged in Nexus. The Maven repository URL is: > * https://repository.apache.org/content/repositories/orgapacheiceberg-1005/ > <https://repository.apache.org/content/repositories/orgapacheiceberg-1005/> > > This release contains many bug fixes and several new features: > * Actions to remove orphaned files and to optimize metadata for query > performance > * Support for ORC data files > * Snapshot cherry-picking > * Incremental scan planning based on table history > * In and notIn expressions > * An InputFormat for writing MR jobs > > Please download, verify, and test. > > Please vote in the next 72 hours. > > [ ] +1 Release this as Apache Iceberg 0.8.0-incubating > [ ] +0 > [ ] -1 Do not release this because... > > > -- > Ryan Blue > > > -- > Ryan Blue > > > -- > Best Regards
