+1 (non-binding) * Verified sha512 checksum was correct for source tarball * Verified signature was correct for source tarball (not verified trust) * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2 * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except ITestS3AContractOpen.testInputStreamReadNegativePosition test which is non-blocker for release as https://issues.apache.org/jira/browse/HADOOP-19556 * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile passing against Amazon S3 in eu-west-1
Suhail, Ahmar <ahma...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52 tarihinde şunu yazdı: > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be > included in the release yet, if the RC fails the vote, will include it in > the next one. > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2...@gmail.com <mailto: > anujmodi2...@gmail.com>> wrote: > > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > > > > Hi, Ahmar > > > Here are the PRs for the fix raised for ABFS close() causing NPE raised on > RC. > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 < > https://github.com/apache/hadoop/pull/7888> > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 < > https://github.com/apache/hadoop/pull/7889> > > > Feel free to merge them as per the release timelines. > > > Thanks > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <ste...@cloudera.com.inva > <mailto:ste...@cloudera.com.inva>lid> > wrote: > > > > just discovering as I do the third party compatibility testing of the > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit > broken > > -nothing to worry about though as its a new feature so not a regression > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahma...@amazon.co.uk.inva > <mailto:ahma...@amazon.co.uk.inva>lid> > > wrote: > > > > > Hey all, > > > > > > I've fixed the following issues: > > > 1) Issues with changelog, release notes and landing page. Created a new > > > build so they now include all the JIRA's that were missing before. > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring > > > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' > > > message when untarring > > > 3) Previous issues with sha512 and tar not being gzipped. > > > > > > RC3 for Hadoop 3.4.2 is now available for voting. > > > > > > The RC is available at: > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> > > > > > > The git tag is release-3.4.2-RC3, commit > > > 84e8b89ee2ebe6923691205b9e171badde7a495c > > > > > > The maven artifacts are staged at: > > > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 < > https://repository.apache.org/content/repositories/orgapachehadoop-1443> > > > > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) > at: > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> > > > > > > > > > This release has been created off of branch-3.4. > > > > > > Thanks, > > > Ahmar > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > <mailto:iwasak...@oss.nttdata.com> > > > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com>>> > wrote: > > > > > > > > > CAUTION: This email originated from outside of the organization. Do not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > > > > > > > > > > > Since I have to regenerate the site and changelog, do I need to do > the > > > whole build again or is there an easier way to do that? (just by > > > regenerating the site manually and updating the artifacts in SVN > maybe?) > > > > > > > > > I think just running the create-release[1] script could be an easy way > > > while it does not provide options for building only site or source > > > tarballs (now). > > > Since the binary tarballs are containing affected CHANGELOG and > > > RELEASENOTES, > > > it would be nice to update the binary tarballs. > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs > > > # could be ok if only documentation is updated. > > > # Signing and checksums must be cared in that case. > > > > > > > > > In addition, I got following warnings when I extracted the binary > > tarballs > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on > > > Ubuntu.:: > > > > > > > > > tar: Ignoring unknown extended header keyword > > > 'LIBARCHIVE.xattr.com.apple.provenance' > > > tar: Ignoring unknown extended header keyword > > > 'LIBARCHIVE.xattr.com.apple.quarantine' > > > tar: Ignoring unknown extended header keyword > > > 'LIBARCHIVE.xattr.com.apple.provenance' > > > tar: Ignoring unknown extended header keyword > > > 'LIBARCHIVE.xattr.com.apple.quarantine' > > > > > > > > > I guess the cause is modifying the tar files on OSX environment. > > > It would be nice to fix too. > > > > > > > > > [1] > > > > > > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > < > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > > > > > < > > > > > > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > < > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > > > > > > > > > > > > > > > Masatake Iwasaki > > > > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote: > > > > Thanks Masatake and Steve, > > > > > > > > Here's the PR for updating the top page of the site > > > https://github.com/apache/hadoop/pull/7887 < > https://github.com/apache/hadoop/pull/7887> < > > > https://github.com/apache/hadoop/pull/7887> < > https://github.com/apache/hadoop/pull/7887>> , if there's anything else > > we > > > think needs to go in there, let me know, otherwise I will merge by > > tomorrow. > > > > > > > > Since I have to regenerate the site and changelog, do I need to do > the > > > whole build again or is there an easier way to do that? (just by > > > regenerating the site manually and updating the artifacts in SVN > maybe?) > > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now. > > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > <mailto:iwasak...@oss.nttdata.com> > > > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com>> > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com> > > > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com>>>> > wrote: > > > > > > > > > > > > CAUTION: This email originated from outside of the organization. Do > not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the update, Ahmar > > > > > > > > > > > > The binary artifacts basically looks good. > > > > I have some comments about inconsistencies. > > > > > > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix > > > Version/s". I updated them. > > > > > > > > > > > > Jira/Git commit message diff starting: > > > ############################################## > > > > Jira not present with version: 3.4.2. Commit: > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS > #7816 > > > > ... > > > > Jira not present with version: 3.4.2. Commit: > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL > > > version to 1.2.1 (#7807) (#7815) > > > > Jira not present with version: 3.4.2. Commit: > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. > LocalDirAllocator > > > still doesn't always recover from directory deletion (#7683) (#7685) > > > > ... > > > > Jira not present with version: 3.4.2. Commit: > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade > > commons-cli > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan. > > > > > > > > > > > > - The top page of site documentation was not updated for 3.4.2. > > > > > > > > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > < > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > > > > < > > > > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > < > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > > > > > > > < > > > > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > < > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > > > > > > > < > > > > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > < > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt > > > > > ;> > > > > > > > > > > > > - Some release artifact are inconsistent as Steve pointed out. > > > > > > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped. > > > > > > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512` > > > > > > > > > > > > Masatake Iwasaki > > > > > > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote: > > > >> Hey all, > > > >> > > > >> Just a reminder about this. Would be great to get some votes in asap > > and > > > >> hopefully finalise 3.4.2. > > > >> > > > >> Thanks, > > > >> Ahmar > > > >> > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org > <mailto:ah...@apache.org> > > <mailto: > > > ah...@apache.org <mailto:ah...@apache.org>> <mailto:ah...@apache.org > <mailto:ah...@apache.org> <mailto:ah...@apache.org <mailto: > ah...@apache.org>>>> > > > wrote: > > > >> > > > >>> Hey all, > > > >>> > > > >>> RC2 for Hadoop 3.4.2 is now available for voting. > > > >>> > > > >>> There are a couple of things to note: > > > >>> > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu > > Chuang > > > for > > > >>> working on the fix. > > > >>> > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK > bundle > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now > > ~600MB, > > > >>> which makes the size of tar > 1GB, and it can no longer be uploaded > > to > > > SVN. > > > >>> For S3A, download SDK bundle v2.29.52 from: > > > >>> > > > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > > < > > > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > > > > > > < > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > ;> > > > < > > > > > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt > > > > > ;>, > > > >>> and drop it into /share/hadoop/common/lib. Release notes will be > > > updated > > > >>> with these instructions. > > > >>> > > > >>> > > > >>> The RC is available at: > > > >>> > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> < > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> < > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> > > > >>> > > > >>> The git tag is release-3.4.2-RC2, commit > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 > > > >>> > > > >>> The maven artifacts are staged at: > > > >>> > > > >>> > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1441 < > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > > < > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> < > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > ;> > > < > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> < > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > ;> > > < > > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > < > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt > > > > > ;> > > > >>> > > > >>> > > > >>> You can find my public key > (02085AFB652F796A3B01D11FD737A6F52281FA98) > > > at: > > > >>> > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> < > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> < > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> > > > >>> > > > >>> > > > >>> This release has been created off of branch-3.4. > > > >>> > > > >>> Thanks, > > > >>> Ahmar > > > >>> > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > <mailto:common-dev-unsubscr...@hadoop.apache.org> > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > <mailto:common-dev-h...@hadoop.apache.org> > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org >
