heguanhui opened a new pull request, #63863:
URL: https://github.com/apache/doris/pull/63863
## Summary
When creating a table in an external catalog (Iceberg/HMS/Paimon/MaxCompute)
via `CREATE TABLE`, the new table is not immediately visible in the internal
meta cache. The `afterCreateTable()` implementations only call
`resetMetaCacheNames()` which clears the `namesCache` but does not populate the
`metaObjCache`. As a result, operations like `SHOW CREATE TABLE` fail with
"table does not exist" until a manual `REFRESH CATALOG` is executed.
The same issue exists in rename table scenarios
(`IcebergMetadataOps.afterRenameTable` and `RefreshManager.replayRefreshTable`
rename branch) where only `resetMetaCacheNames` is called for the new table
name.
## Root Cause
- `afterCreateTable()` in all 4 external catalog implementations
(HiveMetadataOps, IcebergMetadataOps, PaimonMetadataOps, MaxComputeMetadataOps)
only calls `db.resetMetaCacheNames()`, which clears the table name list cache
but does NOT load the new table object into `metaObjCache`
- When subsequent operations call `getTableNullable()` →
`metaCache.getMetaObj()`, the table object is not found → returns null →
reports table does not exist
- The HMS event processor (`MetastoreEventsProcessor`) already handles this
correctly via `registerExternalTableFromEvent()` which calls
`buildTableForInit(checkExists=false)` + `metaCache.updateCache()`, but
`enable_hms_events_incremental_sync` defaults to `false`
## Fix
- Added `registerTableFromCreate(String tblName)` method to
`ExternalDatabase` that builds the table object with `checkExists=false` and
calls `metaCache.updateCache()` to populate both `namesCache` and `metaObjCache`
- Called `registerTableFromCreate` in `ExternalCatalog.createTable()`,
`replayCreateTable()`, and `renameTable()`
- Fixed `RefreshManager.replayRefreshTable()` rename branch: replaced
`resetMetaCacheNames()` with `registerTableFromCreate(newName)`
- Fixed `IcebergMetadataOps.afterRenameTable()`: replaced
`resetMetaCacheNames()` with `registerTableFromCreate(newName)`
## What problem does this PR solve?
Issue Number: close #xxx
Problem Summary: After CREATE TABLE in external catalogs
(Iceberg/HMS/Paimon/MaxCompute), the new table is not visible in cache. Users
must manually run REFRESH CATALOG before they can use the newly created table.
The same issue affects rename table operations. This PR makes newly
created/renamed tables immediately visible by proactively loading them into the
meta cache.
## Release note
Newly created or renamed external catalog tables
(Iceberg/HMS/Paimon/MaxCompute) are now immediately visible without requiring a
manual REFRESH CATALOG.
## Check List (For Author)
- Test: Unit Test + Regression test
- FE unit test: `ExternalDatabaseRegisterTableTest` (6 cases covering
visibility, namesCache update, idempotency, multi-table, replay, isolation)
- Regression test: `external_table_p0` (Iceberg/Hive/Paimon) and
`external_table_p2` (MaxCompute)
- Behavior changed: No (fix only - previously broken scenario now works)
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
