weastel opened a new pull request, #1948:
URL: https://github.com/apache/datafusion-ballista/pull/1948
# Which issue does this PR close?
No linked issue — this is preparatory work for our distributed scheduler HA.
Current Problem is once scheduler is restarted active job cache does not
contain previous job state this causes our concern in HA Scheduler. The
try_acquire_job hook existed on JobState but was never called from TaskManager.
# Rationale for this change
When an executor sends task status updates,
TaskManager::update_task_statuses only processes jobs present in the local
active_job_cache. On a cache miss (e.g. after scheduler failover or when a
different scheduler receives status for a job it did not originally curate),
updates were dropped with an error log and a TODO for “curator changed”
handling.
JobState::try_acquire_job is the intended mechanism for a scheduler to take
over curation of a still-running job. Calling it on cache miss lets a failover
scheduler rehydrate active_job_cache and apply pending task updates, which is
the “acquire-on-update” path described in the HA design doc.
# What changes are included in this PR?
Add get_or_acquire_active_execution_graph on TaskManager: returns the cached
graph if present; otherwise calls JobState::try_acquire_job and inserts the
result into active_job_cache via DashMap::entry().or_insert() (safe under
concurrent acquire attempts).
Update update_task_statuses to use this helper instead of failing
immediately on cache miss.
Downgrade the cache-miss log from error! to warn! when acquisition is not
possible.
No change to the in-memory JobState backend: try_acquire_job still returns
None, so single-scheduler behavior is unchanged until a distributed backend
implements acquisition.
# Are there any user-facing changes?
No user-facing changes. This is internal scheduler behavior only and has no
effect until a JobState implementation that supports try_acquire_job (e.g. etcd
+ shared storage) is added. No API, config, or documentation updates required
for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
