[ 
https://issues.apache.org/jira/browse/SPARK-51606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yang Jie resolved SPARK-51606.
------------------------------
    Fix Version/s: 4.0.0
       Resolution: Fixed

Issue resolved by pull request 50438
[https://github.com/apache/spark/pull/50438]

> After exiting the remote local connect shell, the SparkConnectServer will not 
> terminate.
> ----------------------------------------------------------------------------------------
>
>                 Key: SPARK-51606
>                 URL: https://issues.apache.org/jira/browse/SPARK-51606
>             Project: Spark
>          Issue Type: Bug
>          Components: Connect
>    Affects Versions: 4.0.0, 4.1.0
>            Reporter: Yang Jie
>            Assignee: Hyukjin Kwon
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>
> {code:java}
> bin/spark-shell --remote local
> WARNING: Using incubator modules: jdk.incubator.vector
> Using Spark's default log4j profile: 
> org/apache/spark/log4j2-defaults.properties
> 25/03/26 15:43:55 INFO SparkSession: Spark Connect server started with the 
> log file: 
> /Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/logs/spark-cb51ad74-00e1-4567-9746-3dc9a7888ecb-org.apache.spark.sql.connect.service.SparkConnectServer-1-local.out
> 25/03/26 15:43:56 INFO BaseAllocator: Debug mode disabled. Enable with the VM 
> option -Darrow.memory.debug.allocator=true.
> 25/03/26 15:43:56 INFO DefaultAllocationManagerOption: allocation manager 
> type not specified, using netty as the default type
> 25/03/26 15:43:56 INFO CheckAllocator: Using DefaultAllocationManager at 
> memory/netty/DefaultAllocationManagerFactory.class
> Welcome to
>       ____              __
>      / __/__  ___ _____/ /__
>     _\ \/ _ \/ _ `/ __/  '_/
>    /___/ .__/\_,_/_/ /_/\_\   version 4.1.0-SNAPSHOT
>       /_/
> Type in expressions to have them evaluated.
> Spark connect server version 4.1.0-SNAPSHOT.
> Spark session available as 'spark'.
>    
> scala> exit 
> Bye!
> 25/03/26 15:44:00 INFO ShutdownHookManager: Shutdown hook called
> 25/03/26 15:44:00 INFO ShutdownHookManager: Deleting directory 
> /private/var/folders/j2/cfn7w6795538n_416_27rkqm0000gn/T/spark-ad8dfdf4-cf2b-413f-a9e3-d6e310dff1ea
> bin/spark-shell --remote local
> WARNING: Using incubator modules: jdk.incubator.vector
> Using Spark's default log4j profile: 
> org/apache/spark/log4j2-defaults.properties
> 25/03/26 15:44:04 INFO SparkSession: Spark Connect server started with the 
> log file: 
> /Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/logs/spark-a7b9a1dc-1e16-4e0e-b7c1-8f957d730df3-org.apache.spark.sql.connect.service.SparkConnectServer-1-local.out
> 25/03/26 15:44:05 INFO BaseAllocator: Debug mode disabled. Enable with the VM 
> option -Darrow.memory.debug.allocator=true.
> 25/03/26 15:44:05 INFO DefaultAllocationManagerOption: allocation manager 
> type not specified, using netty as the default type
> 25/03/26 15:44:05 INFO CheckAllocator: Using DefaultAllocationManager at 
> memory/netty/DefaultAllocationManagerFactory.class
> Exception in thread "main" org.apache.spark.SparkException: 
> org.sparkproject.io.grpc.StatusRuntimeException: UNAUTHENTICATED: Invalid 
> authentication token
>       at 
> org.apache.spark.sql.connect.client.GrpcExceptionConverter.toThrowable(GrpcExceptionConverter.scala:162)
>       at 
> org.apache.spark.sql.connect.client.GrpcExceptionConverter.convert(GrpcExceptionConverter.scala:61)
>       at 
> org.apache.spark.sql.connect.client.CustomSparkConnectBlockingStub.analyzePlan(CustomSparkConnectBlockingStub.scala:75)
>       at 
> org.apache.spark.sql.connect.client.SparkConnectClient.analyze(SparkConnectClient.scala:110)
>       at 
> org.apache.spark.sql.connect.client.SparkConnectClient.analyze(SparkConnectClient.scala:256)
>       at 
> org.apache.spark.sql.connect.client.SparkConnectClient.analyze(SparkConnectClient.scala:227)
>       at 
> org.apache.spark.sql.connect.SparkSession.version$lzycompute(SparkSession.scala:92)
>       at 
> org.apache.spark.sql.connect.SparkSession.version(SparkSession.scala:91)
>       at 
> org.apache.spark.sql.application.ConnectRepl$$anon$1.<init>(ConnectRepl.scala:106)
>       at 
> org.apache.spark.sql.application.ConnectRepl$.$anonfun$doMain$1(ConnectRepl.scala:105)
>       at 
> org.apache.spark.sql.connect.SparkSession$.withLocalConnectServer(SparkSession.scala:824)
>       at 
> org.apache.spark.sql.application.ConnectRepl$.doMain(ConnectRepl.scala:67)
>       at 
> org.apache.spark.sql.application.ConnectRepl$.main(ConnectRepl.scala:57)
>       at org.apache.spark.sql.application.ConnectRepl.main(ConnectRepl.scala)
>       at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>       at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
>       at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.base/java.lang.reflect.Method.invoke(Method.java:569)
>       at 
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
>       at 
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1027)
>       at 
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:204)
>       at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:227)
>       at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:96)
>       at 
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1132)
>       at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1141)
>       at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> 25/03/26 15:44:05 INFO ShutdownHookManager: Shutdown hook called
> 25/03/26 15:44:05 INFO ShutdownHookManager: Deleting directory 
> /private/var/folders/j2/cfn7w6795538n_416_27rkqm0000gn/T/spark-17717972-4070-43c7-b306-a060de27a16e
>  
> jps
> 12611 SparkSubmit
> ps -ef | grep 12611            
>   501 12611     1   0  3:43下午 ttys002    0:04.77 
> /Users/yangjie01/Tools/zulu17/bin/java -cp 
> hive-jackson/*:/Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/conf/:/Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/jars/slf4j-api-2.0.17.jar:/Users/yangjie01/Tools/spark-4.1.0-SNAPSHOT-bin-3.4.1/jars/*
>  -Dscala.usejavacp=true -Xmx1g -XX:+IgnoreUnrecognizedVMOptions 
> --add-modules=jdk.incubator.vector 
> --add-opens=java.base/java.lang=ALL-UNNAMED 
> --add-opens=java.base/java.lang.invoke=ALL-UNNAMED 
> --add-opens=java.base/java.lang.reflect=ALL-UNNAMED 
> --add-opens=java.base/java.io=ALL-UNNAMED 
> --add-opens=java.base/java.net=ALL-UNNAMED 
> --add-opens=java.base/java.nio=ALL-UNNAMED 
> --add-opens=java.base/java.util=ALL-UNNAMED 
> --add-opens=java.base/java.util.concurrent=ALL-UNNAMED 
> --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED 
> --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED 
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED 
> --add-opens=java.base/sun.nio.cs=ALL-UNNAMED 
> --add-opens=java.base/sun.security.action=ALL-UNNAMED 
> --add-opens=java.base/sun.util.calendar=ALL-UNNAMED 
> --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED 
> -Djdk.reflect.useDirectMethodHandle=false 
> -Dio.netty.tryReflectionSetAccessible=true --enable-native-access=ALL-UNNAMED 
> -Dderby.connection.requireAuthentication=false 
> org.apache.spark.deploy.SparkSubmit --master local --conf 
> spark.app.submitTime=1742975034128 --conf 
> spark.sql.artifact.isolation.alwaysApplyClassloader=true --conf 
> spark.app.name=Spark shell --conf spark.sql.artifact.isolation.enabled=true 
> --conf spark.submit.deployMode=client --conf 
> spark.ui.showConsoleProgress=true --class 
> org.apache.spark.sql.connect.service.SparkConnectServer --name Spark Connect 
> server spark-internal{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to