ueshin commented on code in PR #49946:
URL: https://github.com/apache/spark/pull/49946#discussion_r1957027015
##########
launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java:
##########
@@ -384,10 +390,9 @@ private List<String> buildPySparkShellCommand(Map<String,
String> env) throws IO
if (remoteStr != null) {
env.put("SPARK_REMOTE", remoteStr);
env.put("SPARK_CONNECT_MODE_ENABLED", "1");
- } else if (conf.getOrDefault(
- SparkLauncher.SPARK_API_MODE,
"classic").toLowerCase(Locale.ROOT).equals("connect") &&
- masterStr != null) {
- env.put("SPARK_REMOTE", masterStr);
+ } else if (isRemote) {
+ // If `removeStr` is not specified but isRemote is true, it means the
api mode is connect.
+ env.put("MASTER", firstNonEmpty(masterStr, "local"));
Review Comment:
So far:
```
% pyspark --remote local
...
>>> spark
<pyspark.sql.connect.session.SparkSession object at 0x10b5bf010>
```
```
% pyspark --conf spark.api.mode=connect --master local
...
pyspark.errors.exceptions.base.PySparkRuntimeError: [CONNECT_URL_NOT_SET]
Cannot create a Spark Connect session because the Spark Connect remote URL has
not been set. Please define the remote URL by setting either the 'spark.remote'
option or the 'SPARK_REMOTE' environment variable.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
