Hi There,
I have cloned spark 3.3.2 snapshot source code where Volcano feature is
present and then I built the binary distribution for the same.
However when I tried to use this spark 3.3.2 SNAPSHOT dist with Volcano but
I have got class not found exception.
--conf
spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
--conf
spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
Exception in thread "main" java.lang.ClassNotFoundException:
org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
at
java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.spark.util.Utils$.classForName(Utils.scala:216)
at
org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.$anonfun$buildFromFeatures$3(KubernetesDriverBuilder.scala:42)
at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
Any help to resolve this issue please ? Basically I would like to use
Volcano from Spark 3.3.2 in Kubernetes cluster.
--
Thanks
Gnana
