github-actions[bot] commented on code in PR #68196:
URL: https://github.com/apache/doris/pull/68196#discussion_r4079341730
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/RefreshManager.java:
##########
@@ -192,6 +213,7 @@ public void replayRefreshTable(ExternalObjectLog log) {
table = db.get().getTableForReplay(log.getTableId());
}
if (!table.isPresent()) {
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateRowCountCache(catalog.getId(),
db.get().getId());
Review Comment:
[P1] Retire engine entries on this cold follower path too. This branch now
fences only row counts and returns, but the 1,000-entry table-object cache can
evict this object while Hive schema/partition/file entries remain independently
resident (file entries keep up to 10,000 items for the 24-hour external-cache
TTL). After a committed insert's refresh log, the follower can therefore
rebuild the table and reuse its pre-insert file listing. This is distinct from
the cold row-count threads: use the canonical DB/table names carried by this
log to route table invalidation before returning (and a conservative engine
DB/catalog fallback for legacy ID-only logs), with a
primed-file-cache/cold-table replay test.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalDatabase.java:
##########
@@ -593,6 +604,9 @@ public void unregisterTable(String tableName) {
// check if the table exists in cache, it not, does return
ExternalTable dorisTable = getTableForReplay(tableName).orElse(null);
if (dorisTable == null) {
+ // The table object cache is much smaller than the row-count
cache. A drop or rename
+ // must still retire stale row counts when the table object has
already been evicted.
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateTable(extCatalog.getId(),
getFullName(), tableName);
Review Comment:
[P1] Preserve the canonical table scope on this cold DROP replay.
`getTableForReplay` resolves mode-1/mode-2 spelling internally, but when the
object is evicted this fallback discards that name and routes the caller
spelling stored in `DropInfo`. Hive schema/partition invalidation compares
local names exactly and file invalidation hashes the supplied names, so a
replay of `mixedtbl` for cached `MixedTbl` clears row counts via the DB
fallback but leaves pre-DROP engine entries reusable after recreation. This is
distinct from the cold-TRUNCATE thread: carry the resolved local name or
invalidate the canonical DB scope here, and add a mixed-case cold follower DROP
test with real Hive entries primed.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalMetaCacheMgr.java:
##########
@@ -681,15 +772,39 @@ public ExternalRowCountCache getRowCountCache() {
}
public void invalidateTableCache(ExternalTable dorisTable) {
- invalidateTable(dorisTable.getCatalog().getId(),
- dorisTable.getDbName(),
- dorisTable.getName());
+ long catalogId = dorisTable.getCatalog().getId();
+ // This path no longer delegates to invalidateTable, so keep Lance's
catalog-wide
+ // access-generation reset; otherwise a refresh can keep serving the
pre-refresh URI.
+ invalidateLanceTableAccess(catalogId);
+ try {
+ routeCatalogEngines(catalogId, cache -> safeInvalidate(
+ cache, catalogId, "invalidateTableCache",
+ () -> cache.invalidateTable(catalogId,
dorisTable.getDbName(), dorisTable.getName())));
+ } finally {
+ invalidateRowCountCache(dorisTable);
Review Comment:
[P1] Fence the held table before releasing its same-name cache slot.
`ExternalDatabase.unregisterTable` removes the local name/id/object first, then
reaches this fence only after routed engine invalidation. A query does not take
the DB write lock, so if the remote name has already been recreated it can load
the replacement in that gap and read the retired row-count entry; both
generations use the same deterministic table ID, and this later fence cannot
retract the plan that already observed it. This is the table-level counterpart
of, but a distinct location from, the catalog/DB recreation threads. Do not
expose the replacement slot until the old count is fenced (or give the
replacement a distinct generation), while retaining completion fencing around
routed invalidation; add a latching DROP/recreate regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
