Re: [Reminder] Spark 3.5 RC Cut

2023-08-01 Thread Emil Ejbyfeldt
Hi, We previously ran some experiments on builds from the 3.5 branch and noticed that Hadoop had a regression (https://issues.apache.org/jira/browse/HADOOP-18568) in their s3a committer affecting 3.3.5 and 3.3.6 (Spark 3.4 uses hadoop 3.3.4). This fix has been merged into Hadoop and will be p

Re: [Reminder] Spark 3.5 RC Cut

2023-08-01 Thread Dongjoon Hyun
Hi, Emil. HADOOP-18568 is still open and it seems to be never a part of the Hadoop trunk branch. Do you mean another JIRA? Dongjoon. On Tue, Aug 1, 2023 at 2:59 AM Emil Ejbyfeldt wrote: > Hi, > > We previously ran some experiments on builds from the 3.5 branch and > noticed that Hadoop had

Re: [Reminder] Spark 3.5 RC Cut

2023-08-01 Thread Emil Ejbyfeldt
Hi, Yes, sorry about that seem to have messed up the link. Should have been https://issues.apache.org/jira/browse/HADOOP-18757 Best, Emil On 01/08/2023 19:08, Dongjoon Hyun wrote: Hi, Emil. HADOOP-18568 is still open and it seems to be never a part of the Hadoop trunk branch. Do you mean

Re: [Reminder] Spark 3.5 RC Cut

2023-08-01 Thread Dongjoon Hyun
It's still invalid information, Emil. Apache Spark is not affected by HADOOP-18757 because it is not a part of both Apache Hadoop 3.3.5 and 3.3.6. HADOOP-18757 seems to be merged just two weeks ago and there is no Apache Hadoop release with it, isn't it? Could you check your local branch once mo

Re: [Reminder] Spark 3.5 RC Cut

2023-08-01 Thread Emil Ejbyfeldt
> Apache Spark is not affected by HADOOP-18757 because it is not a part of > both Apache Hadoop 3.3.5 and 3.3.6. I am not sure I am following what you are trying to say here. Is that the jira is saying that only 3.3.5 is affected? Here I think the Jira is just incorrect. The jira was created (a