yujun777 opened a new issue, #65654: URL: https://github.com/apache/doris/issues/65654
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues) and found no similar issues. ### Version master (reproduced on a local master-based development branch) ### What's Wrong? `LogicalOlapTableStreamScan` can enter an optimizer rewrite loop after partition pruning. The issue is not specific to MTMV itself. MTMV incremental refresh just made it easy to expose, because its generated plan contains stream scans and goes through the same pruning path. The real problem is in the generic stream-scan builder contract used by the optimizer. In the bad case, FE repeatedly rewrites the same stream scan, the query/task does not finish, and FE keeps allocating plans and burning CPU. This issue is related to the Table Stream / row binlog stack tracked by: - #65265 - #65418 ### What You Expected? Once partition pruning has been applied to a stream scan, the rewritten plan should be marked as already pruned, so the same pruning rule should not keep matching it forever. ### How to Reproduce? One easy reproduction path is a stream-based plan produced during IVM regression. For example, `regression-test/suites/mtmv_p0/ivm/test_ivm_partition_unique_key.groovy` can trigger the problem before the fix. The important part is not MTMV semantics themselves, but that the final refresh plan contains `LogicalOlapTableStreamScan` and partition pruning rewrites it. Observed symptoms before the fix: - the task remains `RUNNING` - FE stack traces stay in optimizer rewrite - FE repeatedly hits `PruneOlapScanPartition` on a stream scan ### Current Behavior Observed on FE: - `TopDownVisitorRewriteJob.doRewrite` keeps revisiting the same subtree - `PruneOlapScanPartition` continues matching the same stream scan - the rewritten `LogicalOlapTableStreamScan` is still considered not pruned ### Root Cause `LogicalOlapTableStreamScan.withSelectedPartitionIds(List<Long>, boolean)` does not follow the parent `LogicalOlapScan` contract. In the parent class, the second argument means `hasPartitionPredicate`, and the rebuilt scan is always marked as `partitionPruned=true`. In `LogicalOlapTableStreamScan`, the override interprets the second argument as `isPartitionPruned` and passes it into the constructor directly. As a result, when `PruneOlapScanPartition` calls: ```java scan.withSelectedPartitionIds(prunedPartitions, hasPartitionPredicate) ``` and `hasPartitionPredicate` is `false`, the rebuilt stream scan also gets `partitionPruned=false`. Then the same pruning rule matches again, rewrites again, and the optimizer loops. ### Anything Else? The local fix is to align `LogicalOlapTableStreamScan.withSelectedPartitionIds(List<Long>, boolean)` with the parent contract: - treat the boolean parameter as `hasPartitionPredicate` - always rebuild the stream scan with `partitionPruned=true` After applying that fix locally, the previously hanging reproduction path finishes normally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
