Hello Quanlong Huang, Andrew Sherman, Riza Suminto, Jason Fehr, Joe McDonnell,
Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21803
to look at the new patch set (#40).
Change subject: IMPALA-915: Support cancel queries in frontend
......................................................................
IMPALA-915: Support cancel queries in frontend
Adds support to cancel a query during Frontend planning or metadata
operations. Frontend planning is handled by createExecRequest, so
registers Java Threads executing createExecRequest by their query ID and
provides cancelExecRequest to interrupt the Thread for a particular
query ID.
Cancellation is implemented by setting a boolean for the thread, and
calling Thread.interrupt to trigger InterruptedException from any wait
calls. Several ignored wait calls are updated to check the boolean and
throw an exception if the query has been cancelled, interrupting those
operations. Updates locks to lockInteruptibly. Provides an
implementation of getUninterruptibly that allows cancellation.
Adds periodic checks to the planning process to interrupt planning.
They're primarily useful when planning is waiting on catalogd/HMS. If
planning gets into an algorithmically complex operation, it will not be
interrupted.
Removes check_inflight, as we can now cancel a query before it's
inflight. In the case that cancellation doesn't happen immediately -
because we're in a busy frontend loop that can't be interrupted -
/cancel will block until the frontend reaches an interruption point and
returns to the backend to finalize the query.
When analysis returns, cancellation is finalized in the backend. The
/cancel_query request returns once the query is cancelled. Cancelling
a request can no longer fail, so additional checks for whether the
request has been cancelled before it started executing are added.
Removes setting UpdateQueryStatus when GetExecRequest returns because
that's already handled in ImpalaServer::Execute when it calls
UnregisterQuery in response to an error, and constitutes an update race
on the status with UnregisterQuery triggered by CancelQueryHandler. We
want to use the status from CancelQueryHandler in this case as it
provides more context (about who initiated the cancel); the result of
GetExecRequest is just UserCancelledException. Avoids calling
UnregisterQuery in Execute if the query is already finalized to avoid
redundant "Invalid or unknown query handle" logs.
Extends idle_query_statuses_ to save status for any query interrupted by
an external process - cancelled by a user or timeout - so they can be
handled consistently.
Testing:
- updates test_query_cancel_created to cancel a CREATED query
- added tests to cancel a query while metadata loading is delayed
- removes test_query_cancel_exception, as it no longer demonstrates
relevant behavior; cancelling a query that will encounter an exception
before the exception occurs is no different than other queries
Change-Id: I0d25d4c7fb0b8dcc7dad9510db1e8dca220eeb86
---
M be/src/runtime/query-driver.cc
M be/src/runtime/query-driver.h
M be/src/runtime/runtime-state.h
M be/src/service/child-query.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/service/internal-server.cc
A fe/src/main/java/org/apache/impala/common/UserCancelledException.java
A fe/src/main/java/org/apache/impala/service/Canceller.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java
M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_web_pages.py
M tests/util/web_pages_util.py
M tests/webserver/test_web_pages.py
24 files changed, 741 insertions(+), 226 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/21803/40
--
To view, visit http://gerrit.cloudera.org:8080/21803
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0d25d4c7fb0b8dcc7dad9510db1e8dca220eeb86
Gerrit-Change-Number: 21803
Gerrit-PatchSet: 40
Gerrit-Owner: Michael Smith <michael.sm...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
