Hyukjin Kwon created SPARK-51416:
------------------------------------
Summary: Remove SPARK_CONNECT_MODE when starting Spark Connect
server
Key: SPARK-51416
URL: https://issues.apache.org/jira/browse/SPARK-51416
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 4.0.0
Reporter: Hyukjin Kwon
{code}
SPARK_CONNECT_MODE=1 ./bin/pyspark
{code}
fails as below:
{code}
py4j.protocol.Py4JJavaError: An error occurred while calling
None.org.apache.spark.api.java.JavaSparkContext.
: java.lang.ClassNotFoundException:
org.apache.spark.sql.connect.SparkConnectPlugin
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at
org.apache.spark.util.SparkClassUtils.classForName(SparkClassUtils.scala:41)
at
org.apache.spark.util.SparkClassUtils.classForName$(SparkClassUtils.scala:36)
at org.apache.spark.util.Utils$.classForName(Utils.scala:99)
at
org.apache.spark.util.Utils$.$anonfun$loadExtensions$1(Utils.scala:2828)
at
scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:118)
at
scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:105)
at scala.collection.immutable.ArraySeq.flatMap(ArraySeq.scala:35)
at org.apache.spark.util.Utils$.loadExtensions(Utils.scala:2826)
at
org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:210)
at
org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:196)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:588)
at
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)
at py4j.Gateway.invoke(Gateway.java:238)
at
py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
at
py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:184)
at py4j.ClientServerConnection.run(ClientServerConnection.java:108)
at java.base/java.lang.Thread.run(Thread.java:840)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
