Hi, Emil. HADOOP-18568 is still open and it seems to be never a part of the Hadoop trunk branch.
Do you mean another JIRA? Dongjoon. On Tue, Aug 1, 2023 at 2:59 AM Emil Ejbyfeldt <eejbyfe...@liveintent.com.invalid> wrote: > Hi, > > We previously ran some experiments on builds from the 3.5 branch and > noticed that Hadoop had a regression > (https://issues.apache.org/jira/browse/HADOOP-18568) in their s3a > committer affecting 3.3.5 and 3.3.6 (Spark 3.4 uses hadoop 3.3.4). This > fix has been merged into Hadoop and will be part the next release of > Hadoop. > > From our testing the regression when writing data to S3 with large > number of tasks S3 is severe enough that we would need to revert to > hadoop 3.3.4 in order to use spark 3.5 release. > > Since it only for S3 I am not sure it warrants action changes in Spark > (e.g rolling back hadoop to 3.3.4). But it probably something people > testing the rc against s3 should be aware of. > > Best, > Emil > > On 29/07/2023 10:29, Yuanjian Li wrote: > > Hi everyone, > > > > Following the release timeline, I will cut the RC on*Tuesday, Aug 1st at > > 1 pm PST* as scheduled. > > > > Date Event > > July 17th 2023 > > Late July > > 2023 Code freeze. Release branch cut. > > QA period. Focus on bug fixes, tests, stability and docs. > > Generally, no new features merged. > > > > > > August 2023 Release candidates (RC), voting, etc. until final release > passes > > > > > > Best, > > Yuanjian > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
