Re: Spark on Yarn with Java 17

Dongjoon, thank you for the fast response! Apache Spark 4.0.0 depends on only Apache Hadoop client library. To better understand your answer, does that mean a Spark application built with Java 17 can successfully run on a Hadoop cluster on version 3.3 and Java 8 runtime? On Fri, Dec 8, 2023 at 4

Re: Spark on Yarn with Java 17

Hi, Jason. Apache Spark 4.0.0 depends on only Apache Hadoop client library. You can track all `Apache Spark 4` activities including Hadoop dependency here. https://issues.apache.org/jira/browse/SPARK-44111 (Prepare Apache Spark 4.0.0) According to the release history, the original suggested tim

Spark on Yarn with Java 17

Hi Spark devs, According to the Spark 3.5 release notes, Spark 4 will no longer support Java 8 and 11 (link ). My company is using Spark on Yarn with Java 8 now. When considering a future upgrade to Spark 4, one issue w

[VOTE] Release Spark 3.3.4 (RC1)

Please vote on releasing the following candidate as Apache Spark version 3.3.4. The vote is open until December 15th 1AM (PST) and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.3.4 [ ] -1 Do not release this package because

Re: Apache Spark 3.3.4 EOL Release?

Thank you, Mridul, and Kent, too. Additionally, thank you for volunteering as a release manager, Jungtaek, For the 3.3.4 EOL release, I've already been testing and preparing for one week since my first email. So, why don't you proceed with the Apache Spark 3.5.1 release? It has 142 patches alrea