Hi Ayush, "lean" tar is a small tar file which doesn't contain the AWS SDK because the size of AWS SDK is itself 500 MB. This can ease usage for non AWS users. Even AWS users can add this jar explicitly if desired.
This is created using https://github.com/apache/hadoop-release-support which is the new and easy way to create and validate hadoop releases created by Steve during 3.4.0. Also the link to this is present in [1] . There was a discussion around this during the RC2 testing. I will check with Steve on why we don't have the equivalent for aarch64. [1] https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease On Thu, Oct 17, 2024 at 10:00 PM Ayush Saxena <ayush...@gmail.com> wrote: > What exactly is this "lean tar"? I couldn’t find any mention of it in > [1], nor did I come across any thread establishing consensus on adding > it, as we did for the aarch64 tar. It’s also not clear to me from the > create-release script how it is getting generated. So, where is this > coming from? Additionally, why don’t we have an equivalent for > aarch64? > > -Ayush > > [1] https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease > > On Thu, 17 Oct 2024 at 13:21, Mukund Madhav Thakur > <mtha...@cloudera.com.invalid> wrote: > > > > Adding my +1 (binding) after all the testing. > > Ran a lot of validation workflows from hadoop-release-support all went > > fine. > > Verified checksum and signatures. > > Build orc and reran the failing vectored io tests and it succeeded. > > Build parquet and run the hadoop parquet module tests. > > @Steve Loughran <ste...@apache.org> Build gcs and ran all the tests > which > > went fine. > > > > Also Ahmar is a committer so his vote can be counted as +1 as well. > > > > Thanks everyone for testing! Need just one more binding vote for this to > be > > released. > > > > > > > > > > > > On Thu, Oct 17, 2024 at 10:01 AM Sneha Vijayarajan < > > sneha.vijayara...@gmail.com> wrote: > > > > > +1 (non-binding) > > > > > > Ran the ABFS driver tests and all the tests passed. > > > > > > ============================================================ > > > HNS-OAuth > > > ============================================================ > > > > > > [WARNING] Tests run: 154, Failures: 0, Errors: 0, Skipped: 2 > > > [WARNING] Tests run: 646, Failures: 0, Errors: 0, Skipped: 82 > > > [WARNING] Tests run: 433, Failures: 0, Errors: 0, Skipped: 57 > > > > > > ============================================================ > > > HNS-SharedKey > > > ============================================================ > > > > > > [WARNING] Tests run: 154, Failures: 0, Errors: 0, Skipped: 3 > > > [WARNING] Tests run: 646, Failures: 0, Errors: 0, Skipped: 34 > > > [WARNING] Tests run: 433, Failures: 0, Errors: 0, Skipped: 44 > > > > > > ============================================================ > > > NonHNS-SharedKey > > > ============================================================ > > > > > > [WARNING] Tests run: 154, Failures: 0, Errors: 0, Skipped: 9 > > > [WARNING] Tests run: 630, Failures: 0, Errors: 0, Skipped: 274 > > > [WARNING] Tests run: 433, Failures: 0, Errors: 0, Skipped: 47 > > > > > > ============================================================ > > > AppendBlob-HNS-OAuth > > > ============================================================ > > > > > > [WARNING] Tests run: 154, Failures: 0, Errors: 0, Skipped: 2 > > > [WARNING] Tests run: 646, Failures: 0, Errors: 0, Skipped: 84 > > > [WARNING] Tests run: 433, Failures: 0, Errors: 0, Skipped: 81 > > > > > > Thanks, > > > Sneha Vijayarajan > > > > > > On Thu, Oct 17, 2024 at 8:57 AM Xiaoqiao He <hexiaoq...@apache.org> > wrote: > > > > > > > > Update my vote to +1(binding). > > > > > > > > Addendum: Verified signature was correct for both src/binary/site > > > tarball. > > > > > > > > > > > > On Mon, Oct 14, 2024 at 5:26 PM Xiaoqiao He <hexiaoq...@apache.org> > > > wrote: > > > > > > > > > Thanks Mukund and Steve for driving this release. > > > > > > > > > > +0. Will +1 when signature check passed. > > > > > > > > > > [Y] LICENSE files exist and NOTICE is included. > > > > > [Y] Rat check is ok. mvn clean apache-rat:check > > > > > [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean > > > package > > > > > -DskipTests -Pnative -Pdist -Dtar`. > > > > > [Y] Setup pseudo cluster with HDFS and YARN. > > > > > [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result. > > > > > [Y] Run example mr jobs and check the result - Pi & wordcount. > > > > > [Y] Spot-check and run some unit tests. > > > > > [Y] Skimmed the Web UI of > NameNode/DataNode/Resourcemanager/NodeManager > > > > > etc. > > > > > [Y] Skimmed over the contents of site documentation. > > > > > [Y] Skimmed over the contents of maven repo. > > > > > [N] Verified checksum passed but signature didn't pass. Did you use > > > > > another key to sign? > > > > > > > > > >> hexiaoqiao@hexiaoqiao-ubuntu:~/Public/hadoop-3.4.1$ gpg > --list-key | > > > > >> grep -B2 -A1 "Mukund" > > > > >> pub rsa4096 2023-09-28 [SC] [expires: 2027-09-28] > > > > >> 41C7034C031FB3AF4BBFB5B89F070EBA4202CEC1 > > > > >> uid [ unknown] Mukund Thakur <mtha...@apache.org> > > > > >> sub rsa4096 2023-09-28 [E] [expires: 2027-09-28] > > > > >> hexiaoqiao@hexiaoqiao-ubuntu:~/Public/hadoop-3.4.1$ gpg --verify > > > > >> hadoop-3.4.1.tar.gz.asc hadoop-3.4.1.tar.gz > > > > >> gpg: Signature made 2024年10月10日 星期四 01时10分30秒 CST > > > > >> gpg: using RSA key > > > 53931DAA708291409958BD474D22BB7D32882201 > > > > >> gpg: Can't check signature: No public key > > > > > > > > > > > > > > > Best Regards, > > > > > - He Xiaoqiao > > > > > > > > > > On Fri, Oct 11, 2024 at 1:31 AM Steve Loughran > > > <ste...@cloudera.com.invalid> > > > > > wrote: > > > > > > > > > >> Hney, you did the -lean one too! nice > > > > >> > > > > >> Anyway, yes I'll test. I will try not to find new problems > > > > >> > > > > >> > > > > >> On Thu, 10 Oct 2024 at 12:18, Mukund Madhav Thakur > > > > >> <mtha...@cloudera.com.invalid> wrote: > > > > >> > > > > >> > Apache Hadoop 3.4.1 > > > > >> > > > > > >> > > > > > >> > With help of Steve, I have put together a release candidate > (RC3) > > > for > > > > >> > Hadoop 3.4.1. > > > > >> > > > > > >> > > > > > >> > What we would like is for anyone who can to verify the tarballs, > > > > >> especially > > > > >> > > > > > >> > anyone who can try the arm64 binaries as we want to include them > > > too. > > > > >> > > > > > >> > > > > > >> > The RC is available at: > > > > >> > > > > > >> > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC3/ > > > > >> > > > > > >> > > > > > >> > The git tag is release-3.4.1-RC3, commit > > > > >> > 4d7825309348956336b8f06a08322b78422849b1 > > > > >> > > > > > >> > > > > > >> > The maven artifacts are staged at > > > > >> > > > > > >> > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1430 > > > > >> > > > > > >> > > > > > >> > You can find my public key at: > > > > >> > > > > > >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > >> > > > > > >> > > > > > >> > Change log > > > > >> > > > > > >> > > > > > >> > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC3/CHANGELOG.md > > > > >> > > > > > >> > > > > > >> > Release notes > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC3/RELEASENOTES.md > > > > >> > > > > > >> > > > > > >> > This is off branch-3.4.1 > > > > >> > > > > > >> > > > > > >> > Key changes include > > > > >> > > > > > >> > > > > > >> > * Bulk Delete API. > > > https://issues.apache.org/jira/browse/HADOOP-18679 > > > > >> > > > > > >> > * Fixes and enhancements in Vectored IO API. > > > > >> > > > > > >> > * Improvements in Hadoop Azure connector. > > > > >> > > > > > >> > * Fixes and improvements post upgrade to AWS V2 SDK in > S3AConnector. > > > > >> > > > > > >> > * This release includes Arm64 binaries. Please can anyone with > > > > >> > > > > > >> > compatible systems validate these. > > > > >> > > > > > >> > > > > > >> > Note, because the arm64 binaries are built separately on a > different > > > > >> > > > > > >> > platform and JVM, their jar files may not match those of the x86 > > > > >> > > > > > >> > release -and therefore the maven artifacts. I don't think this > is > > > > >> > > > > > >> > an issue (the ASF actually releases source tarballs, the > binaries > > > are > > > > >> > > > > > >> > there for help only, though with the maven repo that's a bit > > > blurred). > > > > >> > > > > > >> > > > > > >> > The only way to be consistent would actually untar the > x86.tar.gz, > > > > >> > > > > > >> > overwrite its binaries with the arm stuff, retar, sign and push > out > > > > >> > > > > > >> > for the vote. Even automating that would be risky. > > > > >> > > > > > >> > > > > > >> > Please try the release and vote. The vote will run for 5 days. > > > > >> > > > > > >> > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
