[ https://issues.apache.org/jira/browse/SPARK-51606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yang Jie resolved SPARK-51606. ------------------------------ Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50438 [https://github.com/apache/spark/pull/50438] > After exiting the remote local connect shell, the SparkConnectServer will not > terminate. > ---------------------------------------------------------------------------------------- > > Key: SPARK-51606 > URL: https://issues.apache.org/jira/browse/SPARK-51606 > Project: Spark > Issue Type: Bug > Components: Connect > Affects Versions: 4.0.0, 4.1.0 > Reporter: Yang Jie > Assignee: Hyukjin Kwon > Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > {code:java} > bin/spark-shell --remote local > WARNING: Using incubator modules: jdk.incubator.vector > Using Spark's default log4j profile: > org/apache/spark/log4j2-defaults.properties > 25/03/26 15:43:55 INFO SparkSession: Spark Connect server started with the > log file: > /Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/logs/spark-cb51ad74-00e1-4567-9746-3dc9a7888ecb-org.apache.spark.sql.connect.service.SparkConnectServer-1-local.out > 25/03/26 15:43:56 INFO BaseAllocator: Debug mode disabled. Enable with the VM > option -Darrow.memory.debug.allocator=true. > 25/03/26 15:43:56 INFO DefaultAllocationManagerOption: allocation manager > type not specified, using netty as the default type > 25/03/26 15:43:56 INFO CheckAllocator: Using DefaultAllocationManager at > memory/netty/DefaultAllocationManagerFactory.class > Welcome to > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /___/ .__/\_,_/_/ /_/\_\ version 4.1.0-SNAPSHOT > /_/ > Type in expressions to have them evaluated. > Spark connect server version 4.1.0-SNAPSHOT. > Spark session available as 'spark'. > > scala> exit > Bye! > 25/03/26 15:44:00 INFO ShutdownHookManager: Shutdown hook called > 25/03/26 15:44:00 INFO ShutdownHookManager: Deleting directory > /private/var/folders/j2/cfn7w6795538n_416_27rkqm0000gn/T/spark-ad8dfdf4-cf2b-413f-a9e3-d6e310dff1ea > bin/spark-shell --remote local > WARNING: Using incubator modules: jdk.incubator.vector > Using Spark's default log4j profile: > org/apache/spark/log4j2-defaults.properties > 25/03/26 15:44:04 INFO SparkSession: Spark Connect server started with the > log file: > /Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/logs/spark-a7b9a1dc-1e16-4e0e-b7c1-8f957d730df3-org.apache.spark.sql.connect.service.SparkConnectServer-1-local.out > 25/03/26 15:44:05 INFO BaseAllocator: Debug mode disabled. Enable with the VM > option -Darrow.memory.debug.allocator=true. > 25/03/26 15:44:05 INFO DefaultAllocationManagerOption: allocation manager > type not specified, using netty as the default type > 25/03/26 15:44:05 INFO CheckAllocator: Using DefaultAllocationManager at > memory/netty/DefaultAllocationManagerFactory.class > Exception in thread "main" org.apache.spark.SparkException: > org.sparkproject.io.grpc.StatusRuntimeException: UNAUTHENTICATED: Invalid > authentication token > at > org.apache.spark.sql.connect.client.GrpcExceptionConverter.toThrowable(GrpcExceptionConverter.scala:162) > at > org.apache.spark.sql.connect.client.GrpcExceptionConverter.convert(GrpcExceptionConverter.scala:61) > at > org.apache.spark.sql.connect.client.CustomSparkConnectBlockingStub.analyzePlan(CustomSparkConnectBlockingStub.scala:75) > at > org.apache.spark.sql.connect.client.SparkConnectClient.analyze(SparkConnectClient.scala:110) > at > org.apache.spark.sql.connect.client.SparkConnectClient.analyze(SparkConnectClient.scala:256) > at > org.apache.spark.sql.connect.client.SparkConnectClient.analyze(SparkConnectClient.scala:227) > at > org.apache.spark.sql.connect.SparkSession.version$lzycompute(SparkSession.scala:92) > at > org.apache.spark.sql.connect.SparkSession.version(SparkSession.scala:91) > at > org.apache.spark.sql.application.ConnectRepl$$anon$1.<init>(ConnectRepl.scala:106) > at > org.apache.spark.sql.application.ConnectRepl$.$anonfun$doMain$1(ConnectRepl.scala:105) > at > org.apache.spark.sql.connect.SparkSession$.withLocalConnectServer(SparkSession.scala:824) > at > org.apache.spark.sql.application.ConnectRepl$.doMain(ConnectRepl.scala:67) > at > org.apache.spark.sql.application.ConnectRepl$.main(ConnectRepl.scala:57) > at org.apache.spark.sql.application.ConnectRepl.main(ConnectRepl.scala) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:569) > at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1027) > at > org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:204) > at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:227) > at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:96) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1132) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1141) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > 25/03/26 15:44:05 INFO ShutdownHookManager: Shutdown hook called > 25/03/26 15:44:05 INFO ShutdownHookManager: Deleting directory > /private/var/folders/j2/cfn7w6795538n_416_27rkqm0000gn/T/spark-17717972-4070-43c7-b306-a060de27a16e > > jps > 12611 SparkSubmit > ps -ef | grep 12611 > 501 12611 1 0 3:43下午 ttys002 0:04.77 > /Users/yangjie01/Tools/zulu17/bin/java -cp > hive-jackson/*:/Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/conf/:/Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/jars/slf4j-api-2.0.17.jar:/Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/jars/* > -Dscala.usejavacp=true -Xmx1g -XX:+IgnoreUnrecognizedVMOptions > --add-modules=jdk.incubator.vector > --add-opens=java.base/java.lang=ALL-UNNAMED > --add-opens=java.base/java.lang.invoke=ALL-UNNAMED > --add-opens=java.base/java.lang.reflect=ALL-UNNAMED > --add-opens=java.base/java.io=ALL-UNNAMED > --add-opens=java.base/java.net=ALL-UNNAMED > --add-opens=java.base/java.nio=ALL-UNNAMED > --add-opens=java.base/java.util=ALL-UNNAMED > --add-opens=java.base/java.util.concurrent=ALL-UNNAMED > --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED > --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED > --add-opens=java.base/sun.nio.ch=ALL-UNNAMED > --add-opens=java.base/sun.nio.cs=ALL-UNNAMED > --add-opens=java.base/sun.security.action=ALL-UNNAMED > --add-opens=java.base/sun.util.calendar=ALL-UNNAMED > --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED > -Djdk.reflect.useDirectMethodHandle=false > -Dio.netty.tryReflectionSetAccessible=true --enable-native-access=ALL-UNNAMED > -Dderby.connection.requireAuthentication=false > org.apache.spark.deploy.SparkSubmit --master local --conf > spark.app.submitTime=1742975034128 --conf > spark.sql.artifact.isolation.alwaysApplyClassloader=true --conf > spark.app.name=Spark shell --conf spark.sql.artifact.isolation.enabled=true > --conf spark.submit.deployMode=client --conf > spark.ui.showConsoleProgress=true --class > org.apache.spark.sql.connect.service.SparkConnectServer --name Spark Connect > server spark-internal{code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org