Doris-Breakwater commented on issue #68009: URL: https://github.com/apache/doris/issues/68009#issuecomment-5677132531
Breakwater-GitHub-Analysis-Slot: slot_3a49784340ff ## Initial assessment **Confirmed FE bug / implementation gap in `doris-4.0.8-rc02-bc8ea1bac6d` (high confidence).** The supplied stack trace matches the source at the exact reported commit. This issue currently has no labels; `area/iceberg` is the most direct existing project label, with `area/catalog` also applicable. ### Verified code path 1. Nereids correctly identifies the object as an Iceberg view in [`BindRelation`](https://github.com/apache/doris/blob/bc8ea1bac6d62d92cdebd2e3ac33cc668961ec36/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java#L444-L462), obtains its SQL through `getViewText()`/`loadView`, and wraps it in `ExternalView` + `LogicalView`. 2. While deriving the view output, [`LogicalView.computeOutput()`](https://github.com/apache/doris/blob/bc8ea1bac6d62d92cdebd2e3ac33cc668961ec36/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalView.java#L118-L143) asks the wrapper for `getFullSchema()`. [`ExternalView.getFullSchema()`](https://github.com/apache/doris/blob/bc8ea1bac6d62d92cdebd2e3ac33cc668961ec36/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalView.java#L69-L72) delegates to the underlying `IcebergExternalTable`. 3. [`IcebergExternalTable.getFullSchema()`](https://github.com/apache/doris/blob/bc8ea1bac6d62d92cdebd2e3ac33cc668961ec36/fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalTable.java#L256-L259) unconditionally calls `IcebergUtils.getIcebergSchema(this)`. That method obtains a snapshot first, which enters `IcebergMetadataCache.getSnapshotCache()` and its table cache; a cache miss calls `IcebergMetadataOps.loadTable()`. HMS then correctly rejects the `ICEBERG-VIEW` object as not being an Iceberg table. 4. A view-aware schema path already exists: [`IcebergExternalTable.initSchema()`](https://github.com/apache/doris/blob/bc8ea1bac6d62d92cdebd2e3ac33cc668961ec36/fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalTable.java#L91-L95) passes `isView`, and [`IcebergUtils.loadViewSchemaCacheValue()`](https://github.com/apache/doris/blob/bc8ea1bac6d62d92cdebd2e3ac33cc668961ec36/fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergUtils.java#L1602-L1612) loads `view.schema()`. The `getFullSchema()` override bypasses that path. 5. `SHOW CREATE TABLE` succeeds for the same reason described in the report: its view branch calls `showCreateView()` -> `getViewText()` -> `getIcebergView()`, without entering the table snapshot cache. This is more accurately described as incomplete wiring in the original Iceberg-view support than as a demonstrated 4.0.8 regression. The existing HMS Iceberg-view regression suite is [unconditionally disabled with `if (true)`](https://github.com/apache/doris/blob/bc8ea1bac6d62d92cdebd2e3ac33cc668961ec36/regression-test/suites/external_table_p0/iceberg/test_iceberg_view_query_p0.groovy#L20-L28), so the failing `SELECT` path has no active coverage. Source history also shows the `LogicalView` schema access predates the Iceberg-view feature. The Kerberos/HDFS and Ranger warnings are not on the exception path shown above. No evidence currently links them to the `NoSuchIcebergTableException`. ### About the 186-second `lock_tables` value This value alone does **not** prove that table locks were held for 186 seconds. In this version, `collectAndLockTable()` records the phase end only after both relation collection and `statementContext.lock()`, and `Nereids Lock Table Time` is measured from parse completion to that shared endpoint. Slow catalog initialization/metadata lookup during collection and actual lock acquisition are therefore aggregated. The timeout may be related metadata warm-up, lock contention, or both; the current evidence cannot distinguish them. To establish the timeout cause, please provide: - the query ID and complete FE log interval from query receipt through timeout; - one or more FE thread dumps captured while the query is stalled; - the full planning profile/timing JSON, plus any concurrent refresh/DDL activity involving the referenced tables; - the minimal Spark Iceberg view DDL and Spark/Iceberg version (also useful for an enabled regression test). These items are not required to confirm the main schema-loading bug; they are needed only to prove the separate long-planning cause. ### Recommended next steps 1. Add an enabled HMS Iceberg-view regression covering `SELECT`, `DESCRIBE`, and `SHOW CREATE`, including the reported `UNION ALL` shape. Assert that querying a view never invokes the Iceberg table/snapshot loader. 2. In the legacy Iceberg table implementation, make `getFullSchema()` select the existing view-schema cache/load path for `isView()`, while preserving the current snapshot-aware schema path for real tables. A narrow implementation can use the existing `IcebergSchemaCacheKey`/`NEWEST_SCHEMA_ID` view-aware loader rather than duplicating Iceberg schema conversion. 3. Verify that ordinary table queries and table time travel remain unchanged, and test view schema aliases/types against the analyzed child output. 4. Audit maintained legacy branches: current `branch-4.0`, `branch-4.1`, and `branch-4.2` source all still contain an unconditional table-snapshot schema path for `IcebergExternalTable.getFullSchema()`. The plugin-driven master path already obtains view columns from `getViewDefinition()`, but it should still receive an end-to-end HMS view test. The documented workaround - querying the base Iceberg tables or using a Doris-side view - is appropriate until a fixed build is available. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
