this is my PR for parquet buffers https://github.com/apache/hadoop/pull/7732
doesn't do any complex work, just * adds a capability to declare buffers may be sliced * adds a switch to turn verification off for file:// * adds tests to show that slicing may happen * docs For Parquet I'm fixing the tests just by deleting the .crc files; will propose rolling back enabling vector IO PR and document it. Vector reading is essential for s3a perf unless you use the analytics stream, so people should turn it on there. for local FS without checksums, it is really good too as it uses async IO for parallel reads. No difference for other stores. I'm not going to turn it off. On Mon, 9 Jun 2025 at 16:13, Cheng Pan <pan3...@gmail.com> wrote: > Thanks for kicking off this release. > > I integrated it with Spark master branch[1], no surprise so far. Looking > forward to the next RC. > > [1] https://github.com/apache/spark/pull/51127 > > Thanks, > Cheng Pan > > On 2025/05/28 12:24:52 Ahmar Suhail wrote: > > Hey all, > > > > The first release candidate for Hadoop 3.4.2 is now available for voting. > > > > There are a couple of things to note: > > > > 1/ No Arm64 artifacts. This is due to previously reported issues: > > https://issues.apache.org/jira/projects/YARN/issues/YARN-11712 and > > ttps://issues.apache.org/jira/projects/YARN/issues/YARN-11713 > > <https://issues.apache.org/jira/projects/YARN/issues/YARN-11713>, which > > mean that the build fails on arm64. > > > > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle > > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB, > > which makes the size of tar > 1GB, and it can no longer be uploaded to > SVN. > > For S3A, download SDK bundle v2.29.52 from: > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 > , > > and drop it into /share/hadoop/common/lib. Release notes will be updated > > with these instructions. > > > > > > The RC is available at: > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.2-RC1/ > > > > The git tag is release-3.4.2-RC1, commit > > 09870840ec35b48cd107972eb24d25e8aece04c9 > > > > The maven artifacts are staged at: > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1437 > > > > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at: > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > This release has been created off of branch-3.4. Key changes include: > > > > * S3A: Integration with S3 Analytics Accelerator input stream > > * S3A: Support for S3 conditional writes > > * ABFS: Deprecation of WASB driver > > * ABFS: Support for Non-Heirarchical Namespace Accounts on ABFS Driver > > > > > > This is my first attempt at managing a release, please do test the > release > > and let me know in case of any issues. > > > > Thanks, > > Ahmar > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
