Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21096 )
Change subject: IMPALA-12855: Fix NPE in firing RELOAD events when the partition doesn't exist ...................................................................... Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/21096/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21096/1//COMMIT_MSG@26 PS1, Line 26: indentio > typo Done http://gerrit.cloudera.org:8080/#/c/21096/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/21096/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@6799 PS1, Line 6799: List<Long> eventId > Not related to the current bug, but using the events ids generated here loo There are no events generated when we are reloading the table - we just fetch metadata from HMS and the storage (e.g. HDFS). The RELOAD events are generated at this line by MetastoreShim.fireReloadEventHelper(). I think it's not too late to use them. However, it does have a chance that they arrive before we acquire the table lock at L6805. Ideally the operation that holding the table lock could also update the lastRefreshEventId so don't need to update it here. But we do have a risk to process the self RELOAD events. I'm not sure if we have better choice. We can have a follow-up JIRA to improve this. For using the latest event id from HMS as lastRefreshEventId, we already do this when enable_sync_to_latest_event_on_ddls is true: https://github.com/apache/impala/blob/7c53e87aa166bb77cd2e31646ff913302912d3fd/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L2665-L2668 https://github.com/apache/impala/blob/7c53e87aa166bb77cd2e31646ff913302912d3fd/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L2707 But the latest event id we fetched during reload is still smaller than the RELOAD event ids here since they are fired after the reload. -- To view, visit http://gerrit.cloudera.org:8080/21096 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01af3624bf7cf5cd69935cffa28d54f6a6807504 Gerrit-Change-Number: 21096 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Sun, 03 Mar 2024 02:29:18 +0000 Gerrit-HasComments: Yes
