HyukjinKwon commented on PR #49971:
URL: https://github.com/apache/spark/pull/49971#issuecomment-2747821821
This breaks both PySpark shell and Scala shell with Spark Connect (and
therefore also default `bin/pyspark` and `bin/spark-shell` for connect
distribution)
```bash
bin/spark-shell --remote local
```
```pyspark
bin/pyspark --remote local
```
```
WARNING: Using incubator modules: jdk.incubator.vector
Exception in thread "main" java.lang.NoClassDefFoundError:
org/sparkproject/com/google/common/io/BaseEncoding
at org.sparkproject.io.grpc.Metadata.<clinit>(Metadata.java:114)
at
org.apache.spark.sql.connect.client.SparkConnectClient$.<clinit>(SparkConnectClient.scala:407)
at
org.apache.spark.sql.connect.SparkSession$.withLocalConnectServer(SparkSession.scala:780)
at
org.apache.spark.sql.application.ConnectRepl$.doMain(ConnectRepl.scala:67)
at
org.apache.spark.sql.application.ConnectRepl$.main(ConnectRepl.scala:57)
at org.apache.spark.sql.application.ConnectRepl.main(ConnectRepl.scala)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1027)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:204)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:227)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:96)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1132)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1141)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException:
org.sparkproject.com.google.common.io.BaseEncoding
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 18 more
```
I am going to revert this to unblock the release since it seems improving
the test way and removing some duplicate jars but it broke the main entry
points. Let me know if you guys think differently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
