+1 (non-binding)
1. Verified signatures 2. Verified checksums 3. RAT checks 4. Build and test 5. Tested with Spark 3.2, create a table and run a few queries Thanks Szehon On Mon, Jun 6, 2022 at 10:46 AM Daniel Weeks <daniel.c.we...@gmail.com> wrote: > +1 (binding) > > verified sigs/sums/license/build/tests > > As for the detached commit, I believe I commented on this in a prior > release and the parent commit is the head of the 0.13.x branch and the > detached commit is just the version bump, so I'm ok with it, but it sure > would be nice if that wasn't detached. > > -Dan > > On Sun, Jun 5, 2022 at 10:27 PM Kyle Bendickson <k...@tabular.io> wrote: > >> Update: >> >> Running the test suite in IntelliJ that was (and is) having consistent >> test failures via CLI, the issue seems to be resolved. >> So I do think it is indeed a local JVM set up issue. >> >> Investigating the differences now, but the class in question is >> *org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithEngine* >> >> It seems to be caused by a NoClassDefFoundError, specifically for >> org.xerial.snappy.Snappy. It also happens for ORC, but not for parquet. >> >> Included is a sample output: >> ``` >> java.lang.NoClassDefFoundError: Could not initialize class >> org.xerial.snappy.Snappy >> at >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:99) >> ~[snappy-java-1.1.8.jar:1.1.8] >> at >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:91) >> ~[snappy-java-1.1.8.jar:1.1.8] >> at >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:81) >> ~[snappy-java-1.1.8.jar:1.1.8] >> at >> org.apache.tez.common.TezUtils.createByteStringFromConf(TezUtils.java:81) >> ~[tez-api-0.10.1.jar:0.10.1] >> ``` >> >> Apologies for speaking too soon. *I'm now +0 [non-binding] *provided we >> fix the 0.13.x branch and associated commitId to not be in a detached >> state. The tag *apache-iceberg-0.13.2-rc1 *works just fine, but the >> 0.13.x branch doesn't have the commit ID in question. Not sure if that's a >> major concern or not. >> >> Cheers, >> Kyle >> >> On Sun, Jun 5, 2022 at 11:51 AM Kyle Bendickson <k...@tabular.io> wrote: >> >>> Thanks Eduard! >>> >>> I have: >>> - verified the signature >>> - verified the checksum in the file given as well as of the artifact >>> - ran all unit tests on Java 11, all passed >>> - ran all unit tests on Java 8, some hive-3 tests consistently fail (I >>> do notice they passed on Github - but the tests which fail are consistent >>> despite giving the JVM more memory and checking for OOM) >>> - ran a simple smoke test suite of CRUD on namespaces and v1 and v2 >>> tables with Spark (3.2, 3.1) and Flink (1.13 and 1.14). >>> - ran some upsert related tests on Flink 1.13 and 1.14 (1.12 is provided >>> a deprecation notice) >>> >>> *Problems:* >>> I did notice that the *given commit ID is considered unattached (and I >>> wasn't able to check it out).* I am running my tests by using the >>> provided JAR with engines and then running unit tests locally for the >>> commit just prior (with commit ID >>> *fae977a9f0a79266a04647b0df2ab540cf0dcff4*). >>> >>> Not sure if this is a huge issue, but outside of this unattached commit, >>> my only concern is the `iceberg-hive3` failing tests, but as they passed in >>> CI it's possibly an issue with my local setup locally. >>> >>> Running hive-3 test suite alone, the same tests failed multiple times >>> but again might be something to do with my computer / JVM configuration. >>> >>> *I am -1 (non-binding)*, primarily based on the detached commit (as I >>> had quite a good bit of trouble trying to fetch it through my normal >>> processes) as well as the failing hive3 tests (though that's not exactly >>> within my area of expertise). >>> >>> If the hive3 test failures are only something that occurs for me, then >>> if we fix the "Add version.txt commit" in branch 0.3.x such that when I >>> fetch branch 0.3.x it's present, I'd be +1. Unfortunately, I can't help >>> with cleaning up with the release branch outside of advising somebody else >>> (if desired), but I'm happy to help with that. >>> >>> The hive3 test failures for me seem to be OOM related, but I raised my >>> >>> Find attached a picture of the detached commit ID, >>> *0784d64a659abd4fdaa82cdb599a250a7514facf*, per Github. >>> >>> [image: image.png] >>> >>> Example test failures >>> org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithEngine > >>> testCBOWithSelectedColumnsOverlapJoin[fileFormat=AVRO, engine=tez, >>> catalog=HIVE_CATALOG, isVectorized=false] FAILED >>> java.lang.IllegalArgumentException: Failed to execute Hive query >>> 'SELECT c.first_name, o.order_id FROM default.orders o JOIN >>> default.customers c ON o.customer_id = c.customer_id ORDER BY o.order_id >>> DESC': Error while processing statement: FAILED: Execution Error, return >>> code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask >>> at >>> org.apache.iceberg.mr.hive.TestHiveShell.executeStatement(TestHiveShell.java:152) >>> at >>> org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithEngine.testCBOWithSelectedColumnsOverlapJoin(TestHiveIcebergStorageHandlerWithEngine.java:236) >>> >>> Caused by: >>> org.apache.hive.service.cli.HiveSQLException: Error while >>> processing statement: FAILED: Execution Error, return code 1 from >>> org.apache.hadoop.hive.ql.exec.tez.TezTask >>> at >>> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:335) >>> at >>> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226) >>> at >>> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:263) >>> at >>> org.apache.hive.service.cli.operation.Operation.run(Operation.java:247) >>> at >>> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541) >>> at >>> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:510) >>> at >>> org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:267) >>> at >>> org.apache.iceberg.mr.hive.TestHiveShell.executeStatement(TestHiveShell.java:139) >>> ... 1 more >>> >>> Thanks you for working on this, >>> Kyle >>> >>> >>> >>> >>> On Wed, Jun 1, 2022 at 11:12 PM Eduard Tudenhoefner <edu...@dremio.com> >>> wrote: >>> >>>> Hi Everyone, >>>> >>>> I propose that we release the following RC as the official Apache >>>> Iceberg 0.13.2 release. >>>> >>>> The commit ID is *0784d64a659abd4fdaa82cdb599a250a7514facf* >>>> >>>> >>>> - This corresponds to the tag: *apache-iceberg-0.13.2-rc1* >>>> - >>>> https://github.com/apache/iceberg/commits/apache-iceberg-0.13.2-rc1 >>>> - >>>> >>>> https://github.com/apache/iceberg/tree/0784d64a659abd4fdaa82cdb599a250a7514facf >>>> >>>> >>>> The release tarball, signature, and checksums are here: >>>> >>>> - >>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-0.13.2-rc1 >>>> >>>> >>>> You can find the KEYS file here: >>>> >>>> - https://dist.apache.org/repos/dist/dev/iceberg/KEYS >>>> >>>> >>>> Convenience binary artifacts are staged on Nexus. The Maven repository >>>> URL is: >>>> >>>> - >>>> >>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1088/ >>>> >>>> >>>> Please download, verify, and test. >>>> >>>> Please vote in the next 72 hours. >>>> >>>> [ ] +1 Release this as Apache Iceberg <VERSION> >>>> [ ] +0 >>>> [ ] -1 Do not release this because... >>>> >>>
