github-actions[bot] commented on code in PR #69678:
URL: https://github.com/apache/doris/pull/69678#discussion_r4387785573
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/copier/LogicalPlanDeepCopier.java:
##########
@@ -110,6 +111,8 @@ public Plan
visitLogicalCatalogRelation(LogicalCatalogRelation catalogRelation,
catalogRelation.withRelationId(StatementScopeIdGenerator.newRelationId());
if (context.shouldInvalidatePartitionPruning() && newRelation
instanceof LogicalOlapScan) {
newRelation = ((LogicalOlapScan)
newRelation).withPartitionPruned(false);
+ } else if (context.shouldInvalidatePartitionPruning() && newRelation
instanceof LogicalFileScan) {
+ newRelation = ((LogicalFileScan)
newRelation).withPartitionPruned(false);
Review Comment:
[P2] Preserve genuine prune-to-zero state during file-scan copying
Consider `Project(f.id, f.dt, d.payload, LeftJoin(Filter(f.dt IN (f1,f2),
F), Filter(d.k > 2, D), f.id=d.id))`, where Hive D has only partitions k=1/k=2
and a matching join MV partitioned by F has f1 stale and f2 valid/nonempty. D
initially has an empty selection with totalPartitionNum=2. This invalidation
re-runs `PruneFileScanPartition` on the retained empty map, which constructs
`SelectedPartitions(0, {}, true)`.
`PluginDrivenScanNode.resolveRequiredPartitions` interprets empty/total=0 as
unknown and returns scan-all, so the genuine prune-to-zero short circuit is
lost. For Hive with external-table batch mode disabled, `applyFilter` cannot
recover partition pruning for `k > 2` (it extracts EQ/IN only), and
`HiveScanPlanProvider.resolvePartitions` lists every partition and plans its
files/splits. The original scan would have returned `finishPrunedToZeroScan`
immediately. Runtime filtering preserves the LEFT JOIN rows, but choosing the
compensated candidate now incurs full dimension metadata/split planning for a
known-empty input. Preserve the nonempty-universe provenance when re-pruning an
empty retained selection and cover this compensated LEFT JOIN case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
