Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20131 )
Change subject: IMPALA-12152: Add query options to wait for HMS events sync up ...................................................................... Patch Set 37: (10 comments) Please rebase to resolve merge conflict. http://gerrit.cloudera.org:8080/#/c/20131/35//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20131/35//COMMIT_MSG@45 PS35, Line 45: A succeeded wait: : Query Compilation: 937.279ms : - Synced events from Metastore: 909.162ms (909.162ms) : - Metadata of all 1 tables cached: 911.005ms (1.843ms) : - Analysis finished: 919.600ms (8.595ms) : : A failed wait: : Query Compilation: 1s321ms : - Continuing without syncing Metastore events: 40.883ms (40.883ms) : - Metadata load started: 41.618ms (735.633us) > TestEventSyncFailures validates "Continuing without syncing Metastore event Done http://gerrit.cloudera.org:8080/#/c/20131/35/be/src/catalog/catalog-server.cc File be/src/catalog/catalog-server.cc: http://gerrit.cloudera.org:8080/#/c/20131/35/be/src/catalog/catalog-server.cc@103 PS35, Line 103: used in the > The waiting thread in catalogd: https://gerrit.cloudera.org/c/20131/4/fe/sr Done http://gerrit.cloudera.org:8080/#/c/20131/35/be/src/catalog/catalog-server.cc@471 PS35, Line 471: : void WaitForHmsEvent(TWaitForHmsEventResponse& res > Done. Added a histogram metric, impala-server.wait-for-hms-event-durations- Done http://gerrit.cloudera.org:8080/#/c/20131/35/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/20131/35/be/src/service/impala-server.cc@2322 PS35, Line 2322: << " current version: " << catalog_update_info_.catalog_version; : while (catalog_update_info_.catalog_version < catalog_update_version && : > Yeah, that looks better. Done http://gerrit.cloudera.org:8080/#/c/20131/37/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/20131/37/be/src/service/impala-server.cc@2352 PS37, Line 2352: Detected change in catalog service ID Repeated string. This can be a constant and reused. For event key that may repeat multiple times in timeline, it is probably better to also print a number alongside them to distinguish successive event of same kind. For example, also print the catalog_service_id here. http://gerrit.cloudera.org:8080/#/c/20131/35/be/src/service/query-options.h File be/src/service/query-options.h: http://gerrit.cloudera.org:8080/#/c/20131/35/be/src/service/query-options.h@367 PS35, Line 367: QUERY_OPT_FN(sync_hms_events_wait_time_s, SYNC_HMS_EVENTS_WAIT_TIME_S, \ : TQueryOptionLevel::ADVANCED) \ : QUERY_OPT_FN(sync_hms_events_strict_mode, SYNC_HMS_EVENTS_STRICT_MODE, \ : TQueryOptionLevel::ADVANCED) \ > I thought DEVELOPMENT means not GA yet. It seems ADVANCED is more appropria Done http://gerrit.cloudera.org:8080/#/c/20131/35/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/20131/35/tests/custom_cluster/test_events_custom_configs.py@436 PS35, Line 436: f.execute_quer > Extended execute_query_with_hms_sync to support non-strict mode so we can u Done http://gerrit.cloudera.org:8080/#/c/20131/35/tests/custom_cluster/test_kudu.py File tests/custom_cluster/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/20131/35/tests/custom_cluster/test_kudu.py@323 PS35, Line 323: DROP TABLE %s > Modified execute_query_with_hms_sync to invoke execute_query() instead of e Done http://gerrit.cloudera.org:8080/#/c/20131/35/tests/metadata/test_event_processing.py File tests/metadata/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/20131/35/tests/metadata/test_event_processing.py@293 PS35, Line 293: @SkipIfCatalogV2.hms_event_polling_disabled() > Yeah, just to be consistent with other EventProcessor tests. If you launch Ack http://gerrit.cloudera.org:8080/#/c/20131/35/tests/metadata/test_event_processing_base.py File tests/metadata/test_event_processing_base.py: http://gerrit.cloudera.org:8080/#/c/20131/35/tests/metadata/test_event_processing_base.py@30 PS35, Line 30: def _run_test_insert_events_impl(cls, unique_database, is_transactional=False): > Yeah, it's due to the with-clause. It seems after IMPALA-13694, we don't ne Ack -- To view, visit http://gerrit.cloudera.org:8080/20131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36ac941bb2c2217b09fcfa2eb567b011b38efa2a Gerrit-Change-Number: 20131 Gerrit-PatchSet: 37 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Comment-Date: Wed, 19 Feb 2025 01:35:18 +0000 Gerrit-HasComments: Yes
