BTW, the failure was: Job aborted due to stage failure: Exception while getting task result: com.esotericsoftware.kryo.KryoException: java.lang.UnsupportedOperationException Serialization trace: splitOffsets (org.apache.iceberg.GenericDataFile) files (com.netflix.iceberg.spark.source.Writer$TaskCommit)
On Mon, Oct 14, 2019 at 9:35 AM John Zhuge <jzh...@apache.org> wrote: > - Passed all 7 steps > - Build source code at tag apache-iceberg-0.7.0-incubating-rc1 locally, > unit tests passed. However, my downstream Spark 2.3 branch failed > integration tests, possibly due to > https://github.com/apache/incubator-iceberg/issues/446. I will try > Anton's suggestion and report back. > > On Sun, Oct 13, 2019 at 11:48 PM Gautam <gautamkows...@gmail.com> wrote: > >> Ran all steps successfully. >> >> +1 from me. >> >> On Mon, Oct 14, 2019 at 7:30 AM 俊杰陈 <cjjnj...@gmail.com> wrote: >> >>> Ran all steps successfully, +1 >>> >>> On Mon, Oct 14, 2019 at 7:39 AM Ted Gooch <tgo...@netflix.com.invalid> >>> wrote: >>> > >>> > Ran all steps no issues from me. >>> > +1 >>> > >>> > On Sun, Oct 13, 2019 at 12:09 PM Ryan Blue <rb...@netflix.com.invalid> >>> wrote: >>> >> >>> >> +1 (binding) >>> >> >>> >> I went through all of the validation and it looks good. >>> >> >>> >> I also tested the iceberg-spark-runtime Jar with the Apache Spark >>> 2.4.4 download. Copying the runtime Jar into Spark's jars folder works >>> without problems to read and write both path-based tables and Hive tables. >>> Metadata tables work correctly, same with time travel, and metadata tables >>> with time travel also work. I also didn't run out of threads in the test >>> Hive metastore as I did with the last candidate. >>> >> >>> >> On Sun, Oct 13, 2019 at 11:30 AM Anton Okolnychyi < >>> aokolnyc...@apple.com> wrote: >>> >>> >>> >>> +1 from me then >>> >>> >>> >>> On 13 Oct 2019, at 18:33, Ryan Blue <rb...@netflix.com.INVALID> >>> wrote: >>> >>> >>> >>> The publish steps will now sign all of the artifacts, which is >>> required for an Apache release. That's why the publish steps fail in >>> master. To fix this in master, we can come up with a way to only turn on >>> release signatures if `-Prelease` is set, which is how we also select the >>> Apache releases repository. >>> >>> >>> >>> I don't think this is a problem with the release. The convenience >>> binaries in the release must be signed and published from an Apache >>> repository, so this is necessary. If you're trying to use the release, then >>> you don't need to be using JitPack. >>> >>> >>> >>> On Sun, Oct 13, 2019 at 6:53 AM Anton Okolnychyi >>> <aokolnyc...@apple.com.invalid> wrote: >>> >>>> >>> >>>> Verified signature/checksum/rat, run tests. >>> >>>> >>> >>>> No other pending questions except what Arina and Gautam brought up. >>> >>>> >>> >>>> - Anton >>> >>>> >>> >>>> On 13 Oct 2019, at 09:17, Gautam <gautamkows...@gmail.com> wrote: >>> >>>> >>> >>>> I was able to run steps in Ryan's mail just fine but ran into the >>> same thing Arina mentioned .. when running " ./graldew build publish " .. >>> >>>> >>> >>>> A problem was found with the configuration of task >>> ':iceberg-api:signApachePublication'. >>> >>>> > No value has been specified for property 'signatory.keyId'. >>> >>>> >>> >>>> >>> >>>> Something we are expected to do with the keys here? >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> On Sat, Oct 12, 2019 at 8:30 PM Arina Yelchiyeva < >>> arina.yelchiy...@gmail.com> wrote: >>> >>>>> >>> >>>>> Not sure, if this is related to the release vote but after "Update >>> build for Apache releases" commit [1], we are not longer able to build >>> Iceberg using JitPack. >>> >>>>> Error [2]: >>> >>>>> >>> >>>>> * What went wrong: >>> >>>>> A problem was found with the configuration of task >>> ':iceberg-api:signApachePublication'. >>> >>>>> > No value has been specified for property 'signatory.keyId'. >>> >>>>> >>> >>>>> >>> >>>>> [1] >>> https://github.com/apache/incubator-iceberg/commit/2219c86ec6dc5512b2e581f500125841b1b56226 >>> >>>>> [2] >>> https://jitpack.io/com/github/apache/incubator-iceberg/5620f119f4/build.log >>> >>>>> >>> >>>>> On Oct 12, 2019, at 6:08 AM, Julien Le Dem <julien.le...@gmail.com> >>> wrote: >>> >>>>> >>> >>>>> I’m away for a long weekend without my laptop and will be able to >>> try it out on Tuesday (if votes are still needed). >>> >>>>> Happy validation everyone! >>> >>>>> Julien >>> >>>>> >>> >>>>> On Oct 11, 2019, at 18:21, Ryan Blue <b...@apache.org> wrote: >>> >>>>> >>> >>>>> Here are the steps I included on the last thread if you'd like to >>> validate the release: >>> >>>>> >>> >>>>> Download the source tarball, signature (.asc), and checksum >>> (.sha512) >>> >>>>> Import gpg keys: download KEYS and run gpg --import >>> /path/to/downloaded/KEYS (optional if this hasn’t changed) >>> >>>>> Verify the signature by running: gpg --verify >>> apache-iceberg-0.7.0-incubating.tar.gz.asc >>> >>>>> Verify the checksum by running: sha512sum -c >>> apache-iceberg-0.7.0-incubating.tar.gz.sha512 >>> >>>>> Untar the archive and go into the source directory: tar xzf >>> apache-iceberg-0.7.0-incubating.tar.gz && cd apache-iceberg-0.7.0-incubating >>> >>>>> Run RAT checks to validate license headers: dev/check-license >>> >>>>> Build and test the project: ./gradlew build (use Java 8) >>> >>>>> >>> >>>>> You can also validate the LICENSE and NOTICE documentation, which >>> is included in the source tarball, as well as the staged binary artifacts. >>> >>>>> >>> >>>>> To validate the convenience binaries, add the Maven URL from the >>> email above to a downstream project and update your Iceberg dependency to >>> 0.7.0-incubating, like this: >>> >>>>> >>> >>>>> repositories { >>> >>>>> maven { >>> >>>>> name 'stagedIceberg' >>> >>>>> url ' >>> https://repository.apache.org/content/repositories/orgapacheiceberg-1000/ >>> ' >>> >>>>> } >>> >>>>> } >>> >>>>> >>> >>>>> ext { >>> >>>>> icebergVersion = '0.7.0-incubating' >>> >>>>> } >>> >>>>> >>> >>>>> Then run the downstream project’s tests. >>> >>>>> >>> >>>>> Thanks for voting, everyone! >>> >>>>> >>> >>>>> rb >>> >>>>> >>> >>>>> >>> >>>>> On Fri, Oct 11, 2019 at 6:18 PM Ryan Blue <b...@apache.org> wrote: >>> >>>>>> >>> >>>>>> Hi everyone, >>> >>>>>> >>> >>>>>> I propose the following RC to be released as official Apache >>> Iceberg 0.7.0-incubating release. >>> >>>>>> >>> >>>>>> The commit id is 028a8d0e65d9c713b9b040c592fa10641b6c867b >>> >>>>>> * This corresponds to the tag: apache-iceberg-0.7.0-incubating-rc1 >>> >>>>>> * >>> https://github.com/apache/incubator-iceberg/tree/apache-iceberg-0.7.0-incubating-rc1 >>> >>>>>> * >>> https://github.com/apache/incubator-iceberg/tree/028a8d0e65d9c713b9b040c592fa10641b6c867b >>> >>>>>> >>> >>>>>> The release tarball, signature, and checksums are here: >>> >>>>>> * >>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.7.0-incubating-rc1/ >>> >>>>>> >>> >>>>>> You can find the KEYS file here: >>> >>>>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS >>> >>>>>> >>> >>>>>> Convenience binary artifacts are staged in Nexus. The Maven >>> repository URL is: >>> >>>>>> * >>> https://repository.apache.org/content/repositories/orgapacheiceberg-1001/ >>> >>>>>> >>> >>>>>> This is the first Apache Iceberg release. >>> >>>>>> >>> >>>>>> Please download, verify, and test; then vote in the next 72 hours. >>> >>>>>> >>> >>>>>> [ ] +1 Release this as Apache Parquet 0.7.0-incubating >>> >>>>>> [ ] +0 >>> >>>>>> [ ] -1 Do not release this because... >>> >>>>>> >>> >>>>>> -- >>> >>>>>> Ryan Blue >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> -- >>> >>>>> Ryan Blue >>> >>>>> >>> >>>>> >>> >>>> >>> >>> >>> >>> >>> >>> -- >>> >>> Ryan Blue >>> >>> Software Engineer >>> >>> Netflix >>> >>> >>> >>> >>> >> >>> >> >>> >> -- >>> >> Ryan Blue >>> >> Software Engineer >>> >> Netflix >>> >>> >>> >>> -- >>> Thanks & Best Regards >>> >> > > -- > John Zhuge > -- John Zhuge
