Re: 4.0.0-preview1 test report: running on Yarn

2024-06-18 Thread Cheng Pan
FYI, I have submitted SPARK-48651(https://github.com/apache/spark/pull/47010) to update the Spark on YARN docs for JDK configuration, looking forward to your feedback. Thanks, Cheng Pan > On Jun 18, 2024, at 02:00, George Magiros wrote: > > I successfully submitted and ran org.apache.spark.e

Re: 4.0.0-preview1 test report: running on Yarn

2024-06-18 Thread George Magiros
Thank you all so much for the kind words of encouragement on my first test report. As a follow up, I ran all my HDFS and Yarn nodes on Java 8 - including my Nodemanagers. I then modified Spark's conf/spark-defaults.conf according to Mr. Pan's prior post, and it worked: I was able to submit SparkP