Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20473 )
Change subject: IMPALA-11553: Add event specific metrics in the table metrics ...................................................................... Patch Set 7: (2 comments) Thanks for the refactor! http://gerrit.cloudera.org:8080/#/c/20473/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20473/5//COMMIT_MSG@11 PS5, Line 11: identif > MetastoreEventsProcessor.java#1168 -- We are tracking the duration in secon I still don't see where this was changed in the patch. The only metric I sew with 1min/5min/15min is avgNumberOfEventsReceived As far as I remember the slowest event processing I saw in production logs was around 50 minutes (reloading a table with lot of partitions on a slow namenode) http://gerrit.cloudera.org:8080/#/c/20473/5/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/20473/5/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1745 PS5, Line 1745: t type"); > Unfortunately, since the table doesn't exist in cache we cannot see the dur This will also happen with rename, so the duration of rename won't be added to the metric of the new table? I don't think that this is critical, but a Jira could be created + some comments could be added about this. For dropped events there could be a separate metric that collects the sum of duration of all drop events. Generally a drop should be fast, but it takes metastoreDdlLock_, which can block on HMS RPCs. -- To view, visit http://gerrit.cloudera.org:8080/20473 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2428029361e610a0fcd8ed11be2ab771f03b00dd Gerrit-Change-Number: 20473 Gerrit-PatchSet: 7 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Thu, 14 Dec 2023 09:45:58 +0000 Gerrit-HasComments: Yes
