L. C. Hsieh created SPARK-59074:
-----------------------------------
Summary: Import scg-pool-k8s crate into Spark Connect Gateway
Key: SPARK-59074
URL: https://issues.apache.org/jira/browse/SPARK-59074
Project: Spark
Issue Type: Sub-task
Components: Connect
Affects Versions: connect-gateway-0.1.0
Reporter: L. C. Hsieh
Assignee: L. C. Hsieh
Part of importing the Spark Connect Gateway reference implementation into
apache/spark-connect-gateway as a series of small, bottom-up PRs.
This adds the scg-pool-k8s crate — a Kubernetes service-watch backend Pool. It
watches a Service's Endpoints object in a given namespace and exposes the ready
endpoints as the routing backend set, so the gateway's backend membership tracks
the Spark Connect pods dynamically. This is the dynamic counterpart to the
fixed-membership scg-pool-static.
scg-pool-k8s depends on the scg-routing crate (SPARK-58773) whose Pool trait it
implements, and pulls in the kube / k8s-openapi client stack. It is imported as
its own PR (rather than grouped) because of that larger dependency surface. It
compiles and tests on its own (unit tests plus a router integration test) and is
appended to the workspace members list. It is consumed later by the gateway
binary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
