Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22617 )
Change subject: IMPALA-13861: Standardize workload management tests ...................................................................... IMPALA-13861: Standardize workload management tests This patch standardizes tests against workload management tables (sys.impala_query_log and sys.impala_query_live) to use a common superclass named WorkloadManagementTestSuite. The setup_method of this superclass waits for workload management init completion (wait_for_wm_init_complete()), while the teardown_method waits until impala-server.completed-queries.queued metric reaches 0 (wait_for_wm_idle()). test_query_log.py and test_workload_mgmt_sql_details.py are refactored to extend from WorkloadManagementTestSuite. Tests to assert the query log table flush behavior are grouped together in TestQueryLogTableFlush. test_workload_mgmt_sql_details.py::TestWorkloadManagementSQLDetails now uses 1 minicluster instance for all tests. test_workload_mgmt_init.py does not extend from WorkloadManagementTestSuite because it is testing cluster start and restart scenario. This patch only adds wait_for_wm_idle() at teardown_method where it make sense to do so. test_query_live.py does not extend from WorkloadManagementTestSuite because most of its test method require long --query_log_write_interval_s so that DML queries from workload management worker does not disturb sys.impala_query_live. workload_mgmt parameter in CustomClusterTestSuite.with_args() is standardized to setup appropriate default flags in cluster_setup() rather than passing it down to _start_impala_cluster(): IMPALAD_ARGS --enable_workload_mgmt=true --query_log_write_interval_s=1 \ --shutdown_grace_period_s=0 --shutdown_deadline_s=60 and CATALOGD_ARGS --enable_workload_mgmt=true Note that IMPALAD_ARGS and CATALOGD_ARGS flags added by workload_mgmt and impalad_graceful_shutdown parameter are still overridable to different value by explicitly adding it in the impalad_args and catalogd_args parameters. Setting workload_mgmt=True now automatically enables graceful shutdown for the test. Thus, impalad_graceful_shutdown=True is now removed. With beeswax protocol deprecated, this patch also changes the protocol under test from beeswax to hs2. TestQueryLogTableBeeswax is now renamed to TestQueryLogTableBasic. Additionally, print total wait time in wait_for_metric_value(). Testing: - Run modified tests and pass. Change-Id: Iecf6452fa963304e263805ebeb017c843d17dd16 Reviewed-on: http://gerrit.cloudera.org:8080/22617 Reviewed-by: Riza Suminto <riza.sumi...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/service/workload-management-worker.cc M tests/common/custom_cluster_test_suite.py M tests/common/impala_cluster.py M tests/common/impala_service.py A tests/common/wm_test_suite.py M tests/custom_cluster/test_admission_controller.py M tests/custom_cluster/test_query_live.py M tests/custom_cluster/test_query_log.py M tests/custom_cluster/test_workload_mgmt_init.py M tests/custom_cluster/test_workload_mgmt_sql_details.py 10 files changed, 576 insertions(+), 583 deletions(-) Approvals: Riza Suminto: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/22617 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iecf6452fa963304e263805ebeb017c843d17dd16 Gerrit-Change-Number: 22617 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
