Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22793 )
Change subject: IMPALA-13829: Fix waitForHmsEvent response might miss deletions ...................................................................... Patch Set 1: (1 comment) > Patch Set 1: Verified-1 > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/11755/ Let me try rebase http://gerrit.cloudera.org:8080/#/c/22793/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/22793/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@4422 PS1, Line 4422: // Send back the last catalog version that has been propagated to statestore. : // Coordinators can wait for that in case some deletions are missing due to deleteLog : // being GCed. : res.result.setVersion(lastSentTopicUpdate_.get()); This is the critical code block in getCatalogDelta() that ideally should not happen concurrently with waitForHmsEvent() L4411-L4425. // Garbage collect the delete and topic update log. deleteLog_.garbageCollect(ctx.toVersion); topicUpdateLog_.garbageCollectUpdateLogEntries(ctx.toVersion); lastSentTopicUpdate_.set(ctx.toVersion); That code and this setVersion() is not protected by common lock. deleteLog_ iteration in L4411 can also run concurrently with deleteLog_.garbageCollect(). Will that be an issue? Like waitForSyncDdlVersion(), should this just wait for notification from getCatalogDelta()? // Wait for the next topic update. synchronized(topicUpdateLog_) { try { topicUpdateLog_.wait(TOPIC_UPDATE_WAIT_TIMEOUT_MS); } catch (InterruptedException e) { // Ignore } } -- To view, visit http://gerrit.cloudera.org:8080/22793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3ff7bc4d8b47d2aabec0f1fe73a3ab39e8e407cb Gerrit-Change-Number: 22793 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Fri, 18 Apr 2025 16:06:43 +0000 Gerrit-HasComments: Yes
