Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/21655 )
Change subject: IMPALA-13293: Fix too long wait for initial catalog update ...................................................................... Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/21655/3/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/21655/3/fe/src/main/java/org/apache/impala/service/Frontend.java@1882 PS3, Line 1882: * > nit: we can also add CatalogdMetaProvider#isReady() in this comment done, also extended the function comment to mention how some tests do this differently http://gerrit.cloudera.org:8080/#/c/21655/3/fe/src/main/java/org/apache/impala/service/Frontend.java@1889 PS3, Line 1889: while (true) { : // Avoid calling getCatalog again in this attempt as > Wouldn't it be better to wait on the new catalog then? I thought that it is easier to understand this way. By getting the new catalog there is a very small chance to be faster if the catalog is changed between the two getCatalog() calls. But even that is not perfect - the catalog update can arrive between isReady() and waitForCatalogUpdate(), which can lead to an unnecessary wait of 2s if the catalogReadyNotifier_/catalogUpdateEventNotifier_ are already notified when waitForCatalogUpdate() starts to wait for them. This likely to be very rare and (thanks to the timeout) it can only hang for a limited time. I think that the clearer solution would be to refactor this: - add waitForReadyCatalog() and setIsReady () to FeCatalogManager - use waitForReadyCatalog() above instead of waiting for the FeCatalog directly - call FeCatalogManager.setIsReady() above in JniFrontend instead of calling FeCatalog.setIsReady() directly As FeCatalogManager knows when a catalog update arrives, it could avoid the timing issue in the first paragraph. My plan was to make this change minimal, but I can do small refactor if needed. -- To view, visit http://gerrit.cloudera.org:8080/21655 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4da1d0c0d25f80b02c39be863431201d1b913c5d Gerrit-Change-Number: 21655 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Wed, 14 Aug 2024 08:12:01 +0000 Gerrit-HasComments: Yes
