[
https://issues.apache.org/jira/browse/IMPALA-14549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yida Wu updated IMPALA-14549:
-----------------------------
Labels: admission-control resource-management (was: )
> Admission control to use smarter queue scheduling to mitigate large query
> blocking
> ----------------------------------------------------------------------------------
>
> Key: IMPALA-14549
> URL: https://issues.apache.org/jira/browse/IMPALA-14549
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Yida Wu
> Priority: Major
> Labels: admission-control, resource-management
>
> Currently, Impala’s admission control processes queued requests in FIFO
> order. When a large query at the head of the queue can't be admitted due to
> insufficient resources, smaller queries behind it can be blocked for a long
> time, even if they could fit in the available capacity.
> One idea is to enhance the admission scheduler to be smarter by implementing
> backfill scheduling. When the query at the head of the queue is blocked due
> to insufficient resources, the scheduler should scan the queue and allow
> smaller, runnable queries (that fit within currently available resources) to
> execute out of order.
> To prevent starvation of the large head query, this backfill process should
> be constrained. For example, the scheduler could admit one small query, wait
> for it to complete, and then reevaluate the queue. This process would repeat
> until the large query has sufficient resources to be scheduled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
