L. C. Hsieh created SPARK-59103:
-----------------------------------
Summary: Import scg-proxy crate into Spark Connect Gateway
Key: SPARK-59103
URL: https://issues.apache.org/jira/browse/SPARK-59103
Project: Spark
Issue Type: Sub-task
Components: Connect
Affects Versions: connect-gateway-0.1.0
Reporter: L. C. Hsieh
Assignee: L. C. Hsieh
Part of importing the Spark Connect Gateway reference implementation into
apache/spark-connect-gateway as a series of small, bottom-up PRs.
This adds the scg-proxy crate — the gRPC proxy core of the gateway, and the
crate that ties the previously imported layers together. It contains:
- handler: a SparkConnectService implementation that forwards every RPC to a
backend chosen by routing + session affinity, stamping the authenticated
- dial: a per-backend tonic Channel cache, so the dial cost isn't paid per RPC
- outbound: the credentials the gateway presents on the gateway→backend hop
- config_filter: withholds backend-only secrets from Config RPC responses
scg-proxy depends on scg-routing, scg-auth, scg-genproto, scg-observability,
scg-tenant, scg-ratelimit, and scg-audit. Its examples and integration tests
additionally use scg-store-memory, scg-store-redis, scg-pool-static, and
It is imported complete: src, all integration tests (forwarding, auth, audit,
tenant, ratelimit, observability/tracing, multi-tenant e2e, clone-session,
outbound-token), and all examples (ha_smoke, drain_smoke, load, e2e_client,
fake_collector). The whole workspace stays green: cargo clippy --workspace
--all-targets and cargo test --workspace pass (201 tests pass, 10 Redis/
testcontainer tests are #[ignore] so no Docker is needed in CI).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
