raghav-reglobe opened a new pull request, #65989:
URL: https://github.com/apache/doris/pull/65989
### What problem does this PR solve?
Issue Number: close #65988
Related PR: #64933, #64263
Problem Summary:
`information_schema.tables` on an external Iceberg catalog issues one
blocking `loadTable` against the remote catalog for every table in the catalog,
even when the query only projects cheap columns like
`TABLE_SCHEMA`/`TABLE_NAME`. A listing of a few-thousand-table REST catalog
takes minutes and floods the catalog service with per-table GETs (measured:
3,882 tables → 2m52s, 4,365 tables → 5m38s).
This is an interaction of two changes: #64263 made
`IcebergExternalTable.getComment()` read the comment from table metadata (a
remote table load), and #64933's required-columns pruning in `listTableStatus`
guards every status column except the `status.setComment(...)` fill, which
remained unconditional — so the one column whose fill is a remote round-trip is
also the one column that is always computed.
Fix: guard the comment fill with the same
`needTableStatusColumn(requiredColumns, "TABLE_COMMENT")` check used by the
other status columns. Callers that do not project `TABLE_COMMENT` skip the
remote loads entirely; explicit `TABLE_COMMENT` projections keep the #64263
behavior. When `required_columns` is not set (older BE), behavior is unchanged
(fill everything), consistent with the rest of the pruning.
### Release note
Fix information_schema.tables scans on external catalogs loading every
table's metadata from the remote catalog when TABLE_COMMENT is not projected.
### Check List (For Author)
- Test
- [x] Manual test (add detailed scripts or steps below)
Verified on a cluster with Iceberg REST catalogs: with this change,
`SELECT table_schema, table_name FROM information_schema.tables` on a
4,365-table catalog no longer issues per-table loadTable requests (previously
minutes + thousands of catalog GETs); `SELECT table_schema, table_name,
table_comment FROM information_schema.tables` still fetches comments as before.
- Behavior changed:
- [x] No.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
