[ANNOUNCE] Apache Spark 3.5.4 released

We are happy to announce the availability of Spark 3.5.4! Spark 3.5.4 is a maintenance release containing stability fixes. This release is based on the branch-3.5 maintenance branch of Spark. We strongly recommend all 3.5 users to upgrade to this stable release. To download Spark 3.5.4, head over

[VOTE][RESULT] Release Spark 3.5.4 (RC3)

The vote[1] passes with 10 +1s (5 binding +1s). Thanks to all who helped with the release! (* = binding) +1: - Ruifeng Zheng * - Dongjoon Hyun * - Xinrong Meng * - Cheng Pan - Wenchen Fan * - Kaifei Yi - Vlad Rozov - Huaxin Gao * - He Qi - Sakthi +0: None -1: None Bests, Jie Yang [1] https:/

[VOTE] Release Spark 3.5.4 (RC3)

Please vote on releasing the following candidate as Apache Spark version 3.5.4. The vote is open until Dec 20, 10:00:00 UTC and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.5.4 [ ] -1 Do not release this package because ..

[VOTE] Release Spark 3.5.4 (RC2)

Please vote on releasing the following candidate as Apache Spark version 3.5.4. The vote is open until Dec 19, 09:00:00 UTC and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.5.4 [ ] -1 Do not release this package because ..

[VOTE] Release Spark 3.5.4 (RC1)

Please vote on releasing the following candidate as Apache Spark version 3.5.4. The vote is open until Dec 12, 7:00:00 UTC and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.5.4 [ ] -1 Do not release this package because ...

[DISCUSS] Release Apache Spark 3.5.4

Hi dev, It's been approximately 3 months since Sep 9, 2024, when we released version 3.5.3 for branch-3.5. The patchset differing from 3.5.3 has grown significantly, now it contains 57 commits. The JIRA[2] also indicates that more than 45 resolved tickets are aimed at version 3.5.4, including som

Remove HiveContext from Apache Spark 4.0

Hi all, In SPARK-46171 (apache/spark#44077 [1]), I’m trying to remove the deprecated HiveContext from Apache Spark 4.0 since HiveContext has been marked as deprecated after Spark 2.0. This is a long-deprecated API, it should be replaced with SparkSession with enableHiveSupport now, so I think it's

Migrating the Junit framework used in Apache Spark 4.0 from 4.x to 5.x

Hi all, In SPARK-44170 (apache/spark#43074 [1]), I’m trying to migrate the Junit test framework used in Spark 4.0 from Junit4 to Junit5. Although this involves a fair amount of code modifications, given that Junit 4 is still developed based on Java 6 source code and it hasn't released a new vers