[email protected] has posted comments on this change. ( http://gerrit.cloudera.org:8080/21065 )
Change subject: [WIP]IMPALA-12832: Implicit invalidate metadata on event failures ...................................................................... Patch Set 8: (6 comments) Comments fixed on patchset-3 http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2761 PS3, Line 2761: The invalidati > Did you consider using invalidateTableIfExists() instead? Done http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1164 PS3, Line 1164: NotificationEvent e = events.get(events.size() - 1); > I think we need a debug action to simulate throwing exceptions here. IMPALA Have added a inject_process_event_failure_event_types for debugging purpose to inject failures for certain event types randomly. http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1170 PS3, Line 1170: // synchronizing each event processing reduces the scope of the lock so the a : // potential reset() during event processing is not blocked for longer than : // necessary : synchronized (this) { > Can you add a comment that this also includes IllegalStateException from Pr Done. Have added comment. Have used event member function(onFailure()) to handle them. And have added a function canInvalidateTable() to ut these checks. http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1174 PS3, Line 1174: if (eventProcessorStatus_ != EventProcessorStatus.ACTIVE) { > Transaction events don't have the table name but might also be single-table Have handled for abort and commit txn events now. http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1174 PS3, Line 1174: if (eventProcessorStatus_ != EventProcessorStatus.ACTIVE) { > I think that it would be more intuitive to handle this in a member function done. Have used event member function(onFailure()) to handle it now. http://gerrit.cloudera.org:8080/#/c/21065/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1177 PS3, Line 1177: currentEvent_ = event.metastoreNotificationEvent_; > Can we also add a flag to disable this? So if one day we found issues of th Done. Have added process_event_failure and auto_global_invalidate_metadata flags. -- To view, visit http://gerrit.cloudera.org:8080/21065 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia67fc04c995802d3b6b56f79564bf0954b012c6c Gerrit-Change-Number: 21065 Gerrit-PatchSet: 8 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Tue, 27 Feb 2024 09:08:09 +0000 Gerrit-HasComments: Yes
