It's still invalid information, Emil. Apache Spark is not affected by HADOOP-18757 because it is not a part of both Apache Hadoop 3.3.5 and 3.3.6.
HADOOP-18757 seems to be merged just two weeks ago and there is no Apache Hadoop release with it, isn't it? Could you check your local branch once more, please? Dongjoon. On Tue, Aug 1, 2023 at 9:46 PM Emil Ejbyfeldt <eejbyfe...@liveintent.com> wrote: > Hi, > > Yes, sorry about that seem to have messed up the link. Should have been > https://issues.apache.org/jira/browse/HADOOP-18757 > > Best, > Emil > > On 01/08/2023 19:08, Dongjoon Hyun wrote: > > Hi, Emil. > > > > HADOOP-18568 is still open and it seems to be never a part of the Hadoop > > trunk branch. > > > > Do you mean another JIRA? > > > > Dongjoon. > > > > > > > > On Tue, Aug 1, 2023 at 2:59 AM Emil Ejbyfeldt > > <eejbyfe...@liveintent.com.invalid> wrote: > > > > Hi, > > > > We previously ran some experiments on builds from the 3.5 branch and > > noticed that Hadoop had a regression > > (https://issues.apache.org/jira/browse/HADOOP-18568 > > <https://issues.apache.org/jira/browse/HADOOP-18568>) in their s3a > > committer affecting 3.3.5 and 3.3.6 (Spark 3.4 uses hadoop 3.3.4). > This > > fix has been merged into Hadoop and will be part the next release of > > Hadoop. > > > > From our testing the regression when writing data to S3 with large > > number of tasks S3 is severe enough that we would need to revert to > > hadoop 3.3.4 in order to use spark 3.5 release. > > > > Since it only for S3 I am not sure it warrants action changes in > Spark > > (e.g rolling back hadoop to 3.3.4). But it probably something people > > testing the rc against s3 should be aware of. > > > > Best, > > Emil > > > > On 29/07/2023 10:29, Yuanjian Li wrote: > > > Hi everyone, > > > > > > Following the release timeline, I will cut the RC on*Tuesday, Aug > > 1st at > > > 1 pm PST* as scheduled. > > > > > > Date Event > > > July 17th 2023 > > > Late July > > > 2023 Code freeze. Release branch cut. > > > QA period. Focus on bug fixes, tests, stability and docs. > > > Generally, no new features merged. > > > > > > > > > August 2023 Release candidates (RC), voting, etc. until final > > release passes > > > > > > > > > Best, > > > Yuanjian > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > <mailto:dev-unsubscr...@spark.apache.org> > > >
