Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22640 )
Change subject: WIP: Implement in-place reset metadata for CatalogD ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/22640/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/22640/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@a2325 PS2, Line 2325: With this atomic set removed, we should be able to finally reimplement dbCache_ as CatalogObjectCache. http://gerrit.cloudera.org:8080/#/c/22640/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1066 PS2, Line 1066: List<Db> allDbs = new ArrayList<>(getAllDbs()); : allDbs.sort((db1, db2) -> db1.getName().compareTo(db2.getName())); : for (Db db: getAllDbs()) { : // Just peek is OK to not hold write lock. : String currentlyResettingDb = resettingDbs_.peek(); : if (currentlyResettingDb != null : && currentlyResettingDb.compareTo(db.getName().toLowerCase()) <= 0) { : LOG.info("Waiting resetting db " + db.getName() + " to catch up"); : versionLock_.writeLock().lock(); : try { : // name will be lowered in wait method. : waitOngoingResetMetadata(db.getName()); : } finally { versionLock_.writeLock().unlock(); } : } : if (db.isRemoved()) continue; : ctx.numDbs++; : addDatabaseToCatalogDelta(db, ctx); : } We can probably wrap this way of iterating as a new Iterable class, say, InvalidateAwareDbSnapshot. http://gerrit.cloudera.org:8080/#/c/22640/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2401 PS2, Line 2401: versionLock_.writeLock().unlock(); : Thread.sleep(1); : versionLock_.writeLock().lock(); Should measure and log how long reset() is giving up versionLock_.writeLock(). -- To view, visit http://gerrit.cloudera.org:8080/22640 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib4ae2154612746b34484391c5950e74b61f85c9d Gerrit-Change-Number: 22640 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Comment-Date: Thu, 20 Mar 2025 04:29:18 +0000 Gerrit-HasComments: Yes
