k.venureddy2...@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 )
Change subject: IMPALA-12709: Add support for hierarchical metastore event processing ...................................................................... Patch Set 42: (1 comment) http://gerrit.cloudera.org:8080/#/c/21031/39/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/21031/39/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1583 PS39, Line 1583: public synchronized void updateStatus(EventProcessorStatus toStatus) { : eventProcessorStatus_ = toStatus; : } > Need to check for valid EventProcessorStatus transition here. If 2 executors concurrently invoke handleEventProcessException() due to failure in event processing, in both EventProcessorStatus.NEEDS_INVALIDATE or EventProcessorStatus.ERROR cases, if invalidate_global_metadata_on_event_processing_failure flag is true, reset catalog is triggeed and it update the status to pause and reinit the db cache. It may happen that both threads reset catalog. Shouldn't that be ok ? Lets assume first thread update status to EventProcessorStatus.NEEDS_INVALIDATE and the second thread update status to EventProcessorStatus.ERROR. MetastoreEventsProcessor#tryAutoGlobalInvalidateOnFailure() checks for currentStatus == EventProcessorStatus.ERROR or currentStatus == EventProcessorStatus.NEEDS_INVALIDATE before reset. So, if the first thread has triggered catalog reset and it has already updated the status to EventProcessorStatus.PAUSED, then second thread do not trigger the reset catalog again. If the first thread has not updated the status to EventProcessorStatus.PAUSED yet, then second thread also triggers reset catalog. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 42 Gerrit-Owner: Anonymous Coward <k.venureddy2...@gmail.com> Gerrit-Reviewer: Anonymous Coward <cclive1...@gmail.com> Gerrit-Reviewer: Anonymous Coward <k.venureddy2...@gmail.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Comment-Date: Thu, 10 Apr 2025 17:21:00 +0000 Gerrit-HasComments: Yes
