Doris-Breakwater commented on issue #70778:
URL: https://github.com/apache/doris/issues/70778#issuecomment-7029332367

   Breakwater-GitHub-Analysis-Slot: slot_4e6e063fe06c
   
   The reported failure is supported by the FE source at the exact reported 
build commit, `917b20be83878e98f94e3ebf7efccc80346e3e50`. This is a 
schema-endpoint index-selection defect affecting async materialized views. The 
existing minimal reproduction is sufficient for initial triage; no additional 
production logs or query profiles are needed to explain this exception.
   
   **Source-verified mechanism:**
   
   1. [`MetaInfoAction.getTableSchema`, lines 
251–260](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/MetaInfoAction.java#L251-L260)
 initializes `baseId` to `-1` and reads `getBaseIndexId()` only when the table 
type is exactly `OLAP`. Otherwise, `baseId += tbl.getId()` produces `tableId - 
1`.
   2. [`MTMV` extends `OlapTable` but sets its type to 
`MATERIALIZED_VIEW`](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/catalog/MTMV.java#L67-L114),
 so an async MV takes that fallback. For the IDs in this report, the endpoint 
requests index `1795143523870` instead of the actual base index `1795143523872`.
   3. The proc layer uses a different classification: 
[`TableIf.isManagedTable()` includes both `OLAP` and 
`MATERIALIZED_VIEW`](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/catalog/TableIf.java#L625-L627).
 Consequently, 
[`IndexInfoProcDir.lookup()`](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/common/proc/IndexInfoProcDir.java#L121-L130)
 passes the incorrect index ID to `OlapTable.getSchemaByIndexId()`.
   4. 
[`OlapTable.java:1143`](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java#L1138-L1143)
 immediately dereferences `indexIdToMeta.get(indexId)`. A missing entry 
therefore throws the reported NPE before the caller's subsequent `schema == 
null` check can run.
   
   This also explains why SQL `DESC` succeeds: [`DescribeCommand` checks 
`instanceof OlapTable` and reads the real base index 
ID](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DescribeCommand.java#L275-L288),
 which includes `MTMV`. The HTTP 200 response with an application-level error 
is consistent with 
[`ResponseEntityBuilder.internalError()`](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/httpv2/entity/ResponseEntityBuilder.java#L60-L63).
   
   **Suggested next steps for the fix already being worked on:**
   
   - Use the stored `getBaseIndexId()` for managed tables, aligning the REST 
handler with the proc layer. Do not derive it from either `tableId - 1` or 
`tableId + 1`; the observed adjacency is not an API contract.
   - Add endpoint regression coverage for both projection and aggregate async 
MVs, using distinct table and base-index IDs. Assert JSON `code == 0`, the 
view's base-schema entry with `is_base == true`, and field names matching 
`DESC`. HTTP status alone will miss this failure. Retain ordinary OLAP 
coverage, including `with_mv=1` with a synchronous MV/rollup. Setting 
`with_mv=1` cannot bypass the current failure because the base-schema lookup 
happens first.
   - Verify the patch on the reported build/reproduction and check affected 
maintenance branches before selecting backports. This investigation establishes 
the defect in 4.1.5; it does not establish the introduction version or a fixed 
release.
   
   **Scope and remaining evidence:** SQL `DESC` is an available workaround for 
inspecting columns, as already demonstrated in the report. The one-row 
reproduction after successful refresh and the source path above explain the 
field-list failure without ongoing imports or BE spill pressure. There is 
insufficient evidence to attribute the slow PCA queries, repeated full 
refreshes, or spill-limit failure to this endpoint defect. Those need separate 
investigation with query/refresh IDs, SQL and MV definitions, refresh task 
details, corresponding FE/BE logs, and query/refresh profiles.
   
   Verification here was read-only inspection of the exact build's source and 
the issue's reported results. I did not independently run a Doris cluster, 
execute the reproduction, or modify code.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to