Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/21096 )
Change subject: IMPALA-12855: Fix NPE in firing RELOAD events when the partition doesn't exist ...................................................................... Patch Set 2: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/21096/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/21096/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@6799 PS1, Line 6799: List<Long> eventId > There are no events generated when we are reloading the table - we just fet >There are no events generated when we are reloading the table I understand that the events are generated only at this line. My concern is about events generated during the refreshing of the table, e.g. adding a new partition while catalogd reloads file metadata (after getting the partition list from HMS). If these events are processed only after fireReloadEventAndUpdateRefreshEventId() (e.g. because the event processor is lagging), then they will be skipped as their event id is smaller then the reload event's. There is no perfect way to handle modifications that happen in parallel in HMS (at least for external tables), but it seems safer to process events unnecessarily than to skip them incorrectly. -- To view, visit http://gerrit.cloudera.org:8080/21096 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01af3624bf7cf5cd69935cffa28d54f6a6807504 Gerrit-Change-Number: 21096 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Sun, 03 Mar 2024 10:56:57 +0000 Gerrit-HasComments: Yes
