ump45nose opened a new pull request, #2689: URL: https://github.com/apache/datafusion-ballista/pull/2689
# Which issue does this PR close? Closes #2686. # Rationale for this change In `SchedulerState::launch_tasks`, when `get_executor_metadata` fails, the batch's reservations were taken out of the ledger but never refunded as unbound slots. Unlike the hard launch-error branch (which removes the executor and therefore has no budget left to refund), this path leaves the executor registered — for example after a transient miss from a non-memory `ClusterState` backend — so the reserved vcores stayed stuck until the executor re-registered. Before the reservation ledger in #2192 this branch refunded the batch's task count. # What changes are included in this PR? - On the metadata-lookup failure path, return `(executor_id, freed)` from `take_task_reservations`, the same way the rejected-launch path does, so `revive_offers` can `unbind_tasks` and schedule again. - Add `launch_tasks_refunds_vcores_when_executor_metadata_missing` next to the existing launch-rejection tests: bind tasks, drop metadata while leaving available vcores (`InMemoryClusterState::drop_executor_metadata` test helper), launch, and assert the freed slots equal the bound vcores. - `cargo test -p ballista-scheduler --lib launch_tasks_` — 3 passed; `cargo clippy -p ballista-scheduler --lib --tests -- -D warnings` clean; `cargo fmt -p ballista-scheduler` applied. # Are there any user-facing changes? No. --- This PR was prepared with AI assistance; I reviewed the change and ran the checks above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
