OK, thanks. So, I can safely assume that in 3.5.4 SSL is only used for the web UI and history server, right?
What I find a little bit confusing is that there is a namespace called UI and another standalone. The standalone one makes me think that maybe it does more things than just the UI. El mar, 4 feb 2025 3:46 a. m., Aironman DirtDiver <alons...@gmail.com> escribió: > As far as i know, In Apache Spark versions 3.5.x, SSL/TLS encryption for > RPC (Remote Procedure Call) communication is not available. Instead, Spark > utilizes an AES-based encryption mechanism for securing RPC connections. > This method relies on a shared secret and requires RPC authentication to be > enabled. While functional, this approach is considered less secure compared > to SSL/TLS encryption. > > downloads.apache.org > <https://downloads.apache.org/spark/docs/3.5.0/security.html?utm_source=chatgpt.com> > > Starting from Spark version 4.0.0, SSL/TLS encryption for RPC > communications has been introduced as a preferred method over the legacy > AES-based encryption. SSL/TLS is standardized and offers enhanced security. > To enable SSL/TLS encryption in Spark 4.0.0 and later, proper configuration > of keys and certificates is required. It's important to note that SSL/TLS > encryption is not automatically enabled even if spark.ssl.enabled is set; > it must be explicitly configured for RPC communications. > spark.apache.org > <https://spark.apache.org/docs/preview/security.html?utm_source=chatgpt.com> > > Therefore, if you're using Spark 3.5.x, SSL/TLS encryption for RPC > communication is not available. To utilize SSL/TLS encryption, consider > upgrading to Spark version 4.0.0 or later and configuring the necessary SSL > settings. > > The latest stable release of Apache Spark is version 3.5.4, released in > December 2024. The Apache Spark community has been working on version 4.0, > with preview releases made available for testing and feedback. The most > recent preview, Spark 4.0.0-preview2, was announced on September 26, 2024. > > spark.apache.org > <https://spark.apache.org/news/spark-4.0.0-preview2.html?utm_source=chatgpt.com> > > According to the project's planning discussions, the timeline for Spark > 4.0 is as follows: > > - *January 15, 2025*: Code freeze and creation of the branch-4.0. > - *February 1, 2025*: Feature freeze, focusing on bug fixes and > stability improvements. > - *February 15, 2025*: Initiation of the release candidate (RC) > process. > > Given this schedule, the stable release of Spark 4.0 is anticipated in the > first quarter of 2025. > issues.apache.org > <https://issues.apache.org/jira/browse/SPARK-44111?utm_source=chatgpt.com> > > Therefore, SSL/TLS encryption for RPC communication is expected to be > available in the upcoming Spark 4.0 release. In the meantime, for versions > 3.5.x, the AES-based encryption mechanism remains the standard method for > securing RPC communications. > > El lun, 3 feb 2025 a las 22:57, Pablo Fernández (<pablof5...@gmail.com>) > escribió: > >> Is SSL configuration being used for RPC communication in 3.5.* versions? >> >> I am setting up a standalone spark cluster and I am a little bit confused >> in the security configuration. >> >> In the SSL-configuration docs >> <https://spark.apache.org/docs/latest/security.html#ssl-configuration> it >> says that the SSL settings will be use for all the supported communication >> protocols. But this SSL thing is in the web UI section, which makes me >> think that SSL is only for the web UI. >> >> I know that there are spark.network.* configurations that can enable >> AES-based encryption for RPC connections, but I want to understand if >> having ssl and network settings overwrite one or the other? Or if they are >> meant to be used together for better protection? >> >> Thanks >> > > > -- > Alonso Isidoro Roman > [image: https://]about.me/alonso.isidoro.roman > > <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> >
