jerryshao opened a new issue, #13566:
URL: https://github.com/apache/gravitino/issues/13566
### Describe the subtask
`JobManager` fetches every resource of a job (executable, scripts, jars,
files, archives) into the job's staging directory before it hands the job to
the job executor. An executor that runs jobs outside the Gravitino server can't
use those local paths.
Let each job executor decide how to handle job resources:
- Add `JobContext` to the `JobExecutor` SPI. It carries the Gravitino job
id, the metalake, and the job's staging directory. Gravitino still owns the
staging directory's lifecycle.
- Add `JobExecutor#submitJob(JobContext, JobTemplate)`. It receives the
runtime job template with placeholders resolved and the resources left as their
original URIs.
- Its default implementation localizes the template into the staging
directory with `JobResourceUtils#localizeJobTemplate`, then delegates to
`submitJob(JobTemplate)`. Existing executors, including the local executor,
keep working unchanged.
- `submitJob(JobTemplate)` becomes a default method that throws, so a new
executor only needs to implement the context-based method. `JobExecutorFactory`
rejects an executor class that implements neither.
- `JobManager` stops fetching resources and calls `submitJob(JobContext,
JobTemplate)`.
Behavior change: the persisted `runtimeJobTemplate` of a job shows the
resources as their resolved URIs instead of paths in the staging directory.
Depends on #13563.
### Parent issue
#13554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
