[
https://issues.apache.org/jira/browse/SPARK-51223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-51223:
-----------------------------------
Labels: pull-request-available (was: )
> Spark Connect should use ephemeral port for local mode
> ------------------------------------------------------
>
> Key: SPARK-51223
> URL: https://issues.apache.org/jira/browse/SPARK-51223
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: 4.0.0, 3.5.4
> Reporter: Adam Binford
> Priority: Major
> Labels: pull-request-available
>
> Currently Spark Connect will always use the port 15002 by default when
> starting a connect server, no matter how the connect server was started.
> However, for locally started servers, either through --remote local, or the
> new spark.api.mode=connect, a connect server is started purely for the local
> job that is kicked off. If you use --remote local, you can't start more than
> one job without having port conflicts.
> With the connect API mode, this gets worse, as you could kick off two jobs in
> cluster mode in Yarn, and end up with port conflicts on whatever node ends up
> running your driver.
> Locally launched Spark Connect servers like this should just always use
> ephemeral ports to avoid these port conflicts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
