- building HBase 2.4.13 and Hive 3.1.3 against 3.3.5 failed due to
dependency change.
For HBase, classes under com/sun/jersey/json/* and com/sun/xml/* are not
expected in hbase-shaded-with-hadoop-check-invariants.
Updating hbase-shaded/pom.xml is expected to be the fix as done in HBASE-27292.
https://github.com/apache/hbase/commit/00612106b5fa78a0dd198cbcaab610bd8b1be277
[INFO] --- exec-maven-plugin:1.6.0:exec
(check-jar-contents-for-stuff-with-hadoop) @
hbase-shaded-with-hadoop-check-invariants ---
[ERROR] Found artifact with unexpected contents:
'/home/rocky/srcs/bigtop/build/hbase/rpm/BUILD/hbase-2.4.13/hbase-shaded/hbase-shaded-client/target/hbase-shaded-client-2.4.13.jar'
Please check the following and either correct the build or update
the allowed list with reasoning.
com/
com/sun/
com/sun/jersey/
com/sun/jersey/json/
...
For Hive, classes belonging to org.bouncycastle:bcprov-jdk15on:1.68 seem to be
problematic.
Excluding them on hive-jdbc might be the fix.
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
hive-jdbc: Error creating shaded jar: Problem shading JAR
/home/rocky/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.68/bcprov-jdk15on-1.68.jar
entry
META-INF/versions/15/org/bouncycastle/jcajce/provider/asymmetric/edec/SignatureSpi$EdDSA.class:
java.lang.IllegalArgumentException: Unsupported class file major version 59 ->
[Help 1]
...
On 2023/01/02 22:02, Masatake Iwasaki wrote:
Thanks for your great effort for the new release, Steve and Mukund.
+1 while it would be nice if we can address missed Javadocs.
+ verified the signature and checksum.
+ built from source tarball on Rocky Linux 8 and OpenJDK 8 with native profile
enabled.
+ launched pseudo distributed cluster including kms and httpfs with Kerberos
and SSL enabled.
+ created encryption zone, put and read files via httpfs.
+ ran example MR wordcount over encryption zone.
+ built rpm packages by Bigtop and ran smoke-tests on Rocky Linux 8 (both
x86_64 and aarch64).
- building HBase 2.4.13 and Hive 3.1.3 against 3.3.5 failed due to
dependency change.
# while building HBase 2.4.13 and Hive 3.1.3 against Hadoop 3.3.4 worked.
+ skimmed the site contents.
- Javadocs are not contained (under r3.3.5/api).
# The issue can be reproduced even if I built site docs from the source.
Masatake Iwasaki
On 2022/12/22 4:28, Steve Loughran wrote:
Mukund and I have put together a release candidate (RC0) for Hadoop 3.3.5.
Given the time of year it's a bit unrealistic to run a 5 day vote and
expect people to be able to test it thoroughly enough to make this the one
we can ship.
What we would like is for anyone who can to verify the tarballs, and test
the binaries, especially anyone who can try the arm64 binaries. We've got
the building of those done and now the build file will incorporate them
into the release -but neither of us have actually tested it yet. Maybe I
should try it on my pi400 over xmas.
The maven artifacts are up on the apache staging repo -they are the ones
from x86 build. Building and testing downstream apps will be incredibly
helpful.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC0/
The git tag is release-3.3.5-RC0, commit 3262495904d
The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1365/
You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
Change log
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC0/CHANGELOG.md
Release notes
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC0/RELEASENOTES.md
This is off branch-3.3 and is the first big release since 3.3.2.
Key changes include
* Big update of dependencies to try and keep those reports of
transitive CVEs under control -both genuine and false positive.
* HDFS RBF enhancements
* Critical fix to ABFS input stream prefetching for correct reading.
* Vectored IO API for all FSDataInputStream implementations, with
high-performance versions for file:// and s3a:// filesystems.
file:// through java native io
s3a:// parallel GET requests.
* This release includes Arm64 binaries. Please can anyone with
compatible systems validate these.
Please try the release and vote on it, even though i don't know what is a
good timeline here...i'm actually going on holiday in early jan. Mukund is
around and so can drive the process while I'm offline.
Assuming we do have another iteration, the RC1 will not be before mid jan
for that reason
Steve (and mukund)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
