HyukjinKwon commented on PR #49107: URL: https://github.com/apache/spark/pull/49107#issuecomment-2638324887
> Is this supposed to work with cluster deploy mode? Yes. For `--master ...`, It will run a local Spark Connect server where Spark Driver is running. > If so, how would this work on YARN? It looks like it will only use an ephemeral port when running tests, I think YARN would also need to use an ephemeral port? Yes, it will launch a Spark Connect server locally. > And if so, how is the connect server being secured? If this is running in a secured but shared environment, the connect server may be accessible by anyone else using the cluster. This is the same as Py4J server. It does not enable the auth feature by default. We could have the implementation in Spark Connect to but I would prefer to run this separately. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
