zclllyybb commented on issue #65469: URL: https://github.com/apache/doris/issues/65469#issuecomment-4934567698
Breakwater-GitHub-Analysis-Slot: slot_a53c02f00469 This content is generated by AI for reference only. Initial triage for apache/doris#65469: I checked the live issue, the related PR #65468, and the upstream/master code at base commit `e033e2ff8f64da0dc2c36385c5a16a0ecd65b0e4`. The report looks code-consistent and should be treated as a likely FE/Nereids rewrite-order bug, not as a user SQL issue. Why this is plausible: - `BindRelation` creates `LogicalOlapTableStreamScan` for table-stream relations. - `NormalizeOlapTableStreamScan` is the rule that lowers a stream scan into base OLAP/binlog scan plans and exposes the stream virtual columns through parent project aliases. - On current master, `NormalizeOlapTableStreamScan` is registered in the whole-tree rewrite section before CTE inline/pull-up and inside a `notTraverseChildrenOf(LogicalCTEAnchor)` scope. A stream scan in the CTE producer subtree can therefore be hidden from that normalize pass. After CTE inline exposes the producer plan, there is no equivalent normalize pass before the CTE child rewrite pipeline reaches `OperativeColumnDerive`. - `OperativeColumnDerive` derives operative slots from `LogicalOlapScan.getOutput()`. If the stream virtual columns still remain in the scan output, they can be treated as real scan output slots instead of aliases produced by the stream-normalization rule. - There is also a subtype-preservation issue in the same area: `LogicalPlanDeepCopier.visitLogicalCatalogRelation()` calls `withRelationId()` and then `withVirtualColumns()`. `LogicalOlapTableStreamScan` overrides `withRelationId()` on master, but it does not override `withVirtualColumns()`, so dispatch can fall back to `LogicalOlapScan.withVirtualColumns()` and return a plain `LogicalOlapScan` during CTE deep copy/inline. Assessment of PR #65468: The proposed direction is appropriate: preserve `LogicalOlapTableStreamScan` when virtual columns are copied, and move `NormalizeOlapTableStreamScan` into the CTE children rewrite pipeline after partition pruning and before `OperativeColumnDerive`. That matches the dependency order implied by the code: normalization needs pruned stream partitions, and operative-column derivation should see the lowered scan/project shape rather than raw stream virtual output slots. Validation gaps / missing information: - The issue only gives a SQL shape. To make this reproducible, please add the exact base table DDL, stream object creation, and any required session variables/config such as `enable_table_stream` and the IVM/MTMV incremental refresh settings. - Please include the actual IVM planning failure: FE error message or stack trace, the refresh SQL/rewrite SQL if available, and preferably a plan-process or `EXPLAIN` dump showing the stream virtual columns still present on `LogicalOlapTableStreamScan` after CTE rewrite. - PR #65468 currently changes only FE planner code files. Please add a regression/FE test that creates a real stream-scan path under a CTE and asserts planning succeeds with the stream virtual columns lowered into project aliases. The test should cover the issue pattern, not only a direct stream scan. - The PR checks were still pending when inspected, with compile/FE UT/performance not completed yet. Do not close this until those finish and the missing regression coverage question is resolved. Likely impact: This should mainly affect master table-stream/Nereids planning paths where a stream scan is referenced through a CTE, especially IVM incremental refresh planning. Direct stream scans are less likely to hit this exact placement bug because the current whole-tree normalize pass can see them before CTE handling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
