jerryshao opened a new pull request, #13724:
URL: https://github.com/apache/gravitino/pull/13724

   ### What changes were proposed in this pull request?
   
   This is part 1 of 3 of #13613, which adds a job executor named `k8s` that 
runs Spark jobs on Kubernetes. It was split from #13700 to keep each part 
reviewable.
   
   This part adds the module `plugins/k8s-job-executor` and the classes that 
don't depend on how jobs run:
   
   - **Build**: a shaded jar with fabric8 kubernetes-client 7.2.0 and its JDK 
HTTP client. Jackson and the YAML libraries are relocated to 
`org.apache.gravitino.k8s.shaded`. `LICENSE.bin` lists the bundled dependencies.
   - **`K8sJobExecutorConfigs`**: the configurations of the executor, under 
`gravitino.jobExecutor.k8s.`.
   - **`K8sClientUtils`**: creates the Kubernetes client from a kubeconfig and 
context, or from an API server URL with a CA certificate and a token file. The 
token file is read again when it changes. Without either, fabric8's default 
lookup applies, which covers running inside the job cluster.
   - **`K8sJobExecutionId`**: the job execution id, 
`<cluster>/<kind>/<namespace>/<name>`.
   - **`K8sJobResourceUtils`**: the names, labels and annotations of the 
Kubernetes resources created for the jobs.
   
   Nothing uses these classes yet. Part 2 adds the SparkApplication mapping, 
and part 3 (#13700) adds the executor itself.
   
   ### Why are the changes needed?
   
   The only built-in job executor, `local`, runs jobs as child processes of the 
Gravitino server, which isn't suitable for production. #13554 adds an executor 
that runs jobs on Kubernetes and keeps all the job state there.
   
   Part of: #13613
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. The configurations only take effect once the executor is added in part 3 
and registered in #13616.
   
   ### How was this patch tested?
   
   Unit tests in `plugins/k8s-job-executor`.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to