André Schaidhauer Luckmann created SPARK-58642:
--------------------------------------------------
Summary: Support mutual TLS (mTLS) for the Spark Connect gRPC
server
Key: SPARK-58642
URL: https://issues.apache.org/jira/browse/SPARK-58642
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: 4.4.0
Reporter: André Schaidhauer Luckmann
## What
Extend the Spark Connect gRPC server (SPARK-58622 lands the plain TLS piece) to
require and verify a client certificate on incoming connections (mutual TLS /
mTLS), driven by the standard `spark.ssl.connect.*` configuration namespace.
Concretely, when `spark.ssl.connect.needClientAuth=true` the server loads a JKS
trust store from `spark.ssl.connect.trustStore` +
`spark.ssl.connect.trustStorePassword`, wires it into the Netty
`SslContextBuilder` alongside the existing server cert wiring
(`.trustManager(tmf)` + `.clientAuth(ClientAuth.REQUIRE)`), and rejects any
handshake without a trusted client cert.
## Why
Operators wanting mutual authentication between a Connect client and the
Connect server currently have to front the gRPC endpoint with an external
reverse-proxy sidecar (Envoy, nginx, haproxy) doing the mTLS termination. This
is operationally awkward and duplicates configuration a native Spark deployment
already carries in its `spark.ssl.*` namespace. SPARK-58622 closes the
plaintext-server gap for one-way TLS; this ticket closes the same gap for
mutual TLS.
## Scope
- Server-side mTLS only: server requires+verifies client cert.
- JKS trust store for CAs (mirrors `spark.ssl.rpc.trustStore`).
- Fails fast at startup on `needClientAuth=true` + `trustStore` unset.
- Reloading trust manager (`trustStoreReloadingEnabled=true`) is NOT honored in
this PR: server logs a WARN and loads the trust store statically. Full reload
support is a further follow-up.
- Client-side mTLS in Python/Scala Connect clients (client presenting a client
cert) is a separate follow-up.
## Dependency
- Depends on SPARK-58622 (server-side TLS). Reuses the same
`buildConnectSslContext` helper.
## Related
- SPARK-58622 (server-side TLS, prerequisite)
- SPARK-41917 (client-focused TLS on JVM client, still open)
- SPARK-42533 (Scala client SSL, merged)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
