github-actions[bot] commented on code in PR #68196:
URL: https://github.com/apache/doris/pull/68196#discussion_r4069522968
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalMetaCacheMgr.java:
##########
@@ -681,15 +761,36 @@ public ExternalRowCountCache getRowCountCache() {
}
public void invalidateTableCache(ExternalTable dorisTable) {
- invalidateTable(dorisTable.getCatalog().getId(),
- dorisTable.getDbName(),
- dorisTable.getName());
+ long catalogId = dorisTable.getCatalog().getId();
Review Comment:
[P1] Preserve Lance table-access invalidation on this held-table path.
Before this change `invalidateTableCache` delegated to `invalidateTable`, whose
first step calls `invalidateLanceTableAccess`; this direct routing skips that
hook. `REFRESH TABLE` and resident drop/rename reach this method, while Lance
caches the dataset URI and storage options (60-second default TTL), so a
successful refresh can keep opening the pre-refresh URI/options. Call the Lance
hook here (or share the old side effects) before routing, and add a
primed-access regression.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/HiveInsertExecutor.java:
##########
@@ -88,9 +88,25 @@ protected void doAfterCommit() throws DdlException {
List<String> modifiedPartNames = Lists.newArrayList();
List<String> newPartNames = Lists.newArrayList();
if (hmsTable.isPartitionedTable() && partitionUpdates != null &&
!partitionUpdates.isEmpty()) {
- HiveExternalMetaCache cache =
Env.getCurrentEnv().getExtMetaCacheMgr()
- .hive(hmsTable.getCatalog().getId());
- cache.refreshAffectedPartitions(hmsTable, partitionUpdates,
modifiedPartNames, newPartNames);
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateRowCountCache(hmsTable);
Review Comment:
[P1] Extend this held-table fence and recovery boundary over the branch
decision and full-table path. The transaction is already `COMMITTED`, but
`isPartitionedTable()` runs before this fence and can reinitialize/throw; the
non-partitioned or empty-update `invalidateTableCache` call is also outside the
catch, so an engine invalidation failure skips the follower refresh log and
enters `onFail`/rollback after commit. This is distinct from the existing
selective-partition failure thread. Fence at method entry and make both
branches fall through to a conservative full-refresh log, with throwing
predicate/full-invalidation tests.
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/RefreshManager.java:
##########
@@ -274,6 +297,10 @@ public void refreshPartitions(String catalogName, String
dbName, String tableNam
if (!(catalog instanceof ExternalCatalog)) {
throw new DdlException("Only support ExternalCatalog");
}
+ // Partition events are already committed remotely. Fence the cache by
cached identity
+ // before any database/table reload can fail and make the
ignored-not-found path return.
+ Env.getCurrentEnv().getExtMetaCacheMgr()
Review Comment:
[P1] Apply this pre-lookup fence to whole-table HMS events too. Non-rename
`ALTER_TABLE` and non-partitioned `INSERT` call
`refreshExternalTableFromEvent`, which rebuilds an evicted table before any
row-count fence and returns normally when that fallible reload yields null. The
event processor then advances `lastSyncedEventId`; because table-object
eviction has no row-count removal listener, the independent pre-event count
remains resident. This is distinct from the partition-event thread because
these callers never enter `refreshPartitions`; fence their name-based scope
before reload and add cold-table reload-failure coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
