Hello Andrew Sherman, Yida Wu, Abhishek Rawat, Jason Fehr, Joe McDonnell, Csaba
Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22362
to look at the new patch set (#16).
Change subject: IMPALA-13682: Implement missing capabilities in
ImpylaHS2Connection
......................................................................
IMPALA-13682: Implement missing capabilities in ImpylaHS2Connection
This patch implements 'wait_for_finished_timeout',
'wait_for_admission_control', and 'get_admission_result' for
ImpylaHS2Client.
This patch also changes the behavior of ImpylaHS2Connection to produce
several extra cursors aside from self.__cursor for 'execute' call that
supplies user argument and each 'execute_async' to make issuing multiple
concurrent queries possible. Note that each HS2 cursor opens its own HS2
Session. Therefore, this patch breaks the assumption that an
ImpylaHS2Connection is always under a single HS2 Session (see HIVE-11402
and HIVE-14247 on why concurrent query with shared HS2 Session is
problematic). However, they do share the same query options stored at
self.__query_options. In practice, most Impala tests do not care about
running concurrent queries under a single HS2 session but only require
them to use the same query options.
The following additions are new for both BeeswaxConnection and
ImpylaHS2Connection:
- Add method 'log_client' for convenience.
- Implement consistent query state mapping and checking through several
accessor methods.
- Add methods 'wait_for_impala_state' and 'wait_for_any_impala_state'
that use 'get_impala_exec_state' method internally.
- Add 'fetch_profile_after_close' parameter to 'close_query' method. If
True, 'close_query' will return the query profile after closing the
query.
- Add 'discard_results' parameter for 'fetch' method. This can save time
parsing results if the test does not care about the result.
Reuse existing op_handle_to_query_id and add new
session_handle_to_session_id to parse HS2
TOperationHandle.operationId.guid and TSessionHandle.sessionId.guid
respectively.
To show that ImpylaHS2Connection is on par with BeeswaxConnection, this
patch refactors test_admission_controller.py to test using HS2 protocol
by default. Test that does raw HS2 RPC (require capabilities from
HS2TestSuite) is separated out into a new TestAdmissionControllerRawHS2
class and stays using beeswax protocol by default. All calls to
copy.copy is replaced with copy.deepcopy for safety.
Testing:
- Pass exhaustive tests.
Change-Id: I9ac07732424c16338e060c9392100b54337f11b8
---
M be/src/service/client-request-state.h
M tests/beeswax/impala_beeswax.py
M tests/common/impala_connection.py
M tests/common/impala_service.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_admission_controller.py
M tests/custom_cluster/test_session_expiration.py
M tests/util/thrift_util.py
8 files changed, 680 insertions(+), 305 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/22362/16
--
To view, visit http://gerrit.cloudera.org:8080/22362
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9ac07732424c16338e060c9392100b54337f11b8
Gerrit-Change-Number: 22362
Gerrit-PatchSet: 16
Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wydbaggio...@gmail.com>
