github-actions[bot] commented on code in PR #67940:
URL: https://github.com/apache/doris/pull/67940#discussion_r4068732884
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/SubExprAnalyzer.java:
##########
@@ -125,9 +140,70 @@ public Expression visitInSubquery(InSubquery expr, T
context) {
AnalyzedResult analyzedResult = analyzeSubquery(expr);
checkOutputColumn(analyzedResult.getLogicalPlan());
- checkNoCorrelatedSlotsUnderAgg(analyzedResult);
+ // the correlated predicate of an IN subquery may sit below the
aggregation of the subquery:
+ // the rewrite which unnests it (UnCorrelatedApplyAggregateFilter)
computes the aggregation
+ // of the domain of every outer row, the empty correlated domain
included, so that the value
+ // which the IN compares exists for every outer row
+ if (analyzedResult.isCorrelated()) {
+ // The rewrite only carries the outer slots through the filters of
the subquery: it keeps
+ // the aggregation of the domain as it is (the outer predicate
becomes the condition
+ // which pairs the outer row with the rows of the domain) and it
reads the value which
+ // the IN compares from the aggregation itself. An outer slot
which the subquery reads
+ // from its aggregation, its projections or its joins is therefore
rejected here, the way
+ // the scalar subquery path rejects it (see visitScalarSubquery):
the subquery of
+ //
+ // select k from o where k in (select sum(i.v + o.k) from i)
+ //
+ // cannot be unnested, because the aggregation of the domain of an
outer row would have
+ // to aggregate the value of the outer row as well, and the plan
of the rewrite would
+ // read that value from a scan which does not produce it.
+
validateTheNodesOfTheSubqueryReadTheOuterSlotsThroughFilters(analyzedResult.getLogicalPlan(),
+ new
CorrelatedSlotsValidator(ImmutableSet.copyOf(analyzedResult.correlatedSlots)));
+ if
(containsAWindowAboveTheCorrelatedPredicate(analyzedResult.getLogicalPlan(),
+ ImmutableSet.copyOf(analyzedResult.correlatedSlots))) {
+ // The rewrite reads the value which the IN compares from the
aggregation of the domain
+ // of an outer row (the aggregation of the rewrite groups the
rows of one correlation
+ // key), so the nodes of the subquery which sit above the
correlated predicate are
+ // evaluated on the rows of one domain. A window is evaluated
on the rows of the node
+ // it sits in, so a window above the correlated predicate of
the rewrite is evaluated
+ // over the rows of every correlation key together, while that
window of the subquery
+ // of the query is evaluated over the rows of one domain: the
subquery of
+ //
+ // select k from o where k in (
+ // select sum(i.g) over () from i where i.k = o.k
group by i.g)
+ //
+ // is reported as unsupported for that reason. A window below
the correlated
+ // predicate is evaluated before that predicate selects the
rows of the domain in the
+ // plan of the query as well, so the rewrite leaves its
evaluation domain unchanged
+ // and the subquery of
+ //
+ // select k from o where k in (
+ // select rn from (select k, row_number() over (order
by k) as rn from i) x
+ // where x.k = o.k)
+ //
+ // is accepted.
+ throw new AnalysisException(
+ "access outer query's column before window function is
not supported "
+ + analyzedResult.getLogicalPlan());
+ }
+ }
checkNoCorrelatedSlotsUnderSetOp(analyzedResult);
checkRootIsLimit(analyzedResult);
+ if (analyzedResult.isCorrelated()) {
+ // The nodes above the correlated predicate which the rewrites
cannot rebuild per
+ // correlation key are not reported by checkRootIsLimit (it reads
the root of the plan
+ // alone) nor by the validator (it validates the nodes which read
the outer slots):
+ // report them here, the plan of the rewrite would read the
columns of the outer query
+ // from the rows of another correlation key.
+
rejectTheWrappersWhichTheRewriteCannotRebuild(analyzedResult.getLogicalPlan(),
Review Comment:
[P1] Reject a Join above a correlated filter on this widened IN path. For
example, `o.k IN (SELECT count(*) FROM (SELECT i.id,i.k FROM i WHERE i.k=o.k) x
JOIN j ON x.id=j.id)` analyzes as `Apply(IN) -> Aggregate -> Join -> Project ->
Filter(i.k=o.k)`. This validator permits the Filter, and
`rejectTheWrappersWhichTheRewriteCannotRebuild` does not reject the Join on its
path. `locateAggregate` then returns empty at that Join, while the other
uncorrelation rules only reach a Filter/Project at the Apply root.
`InApplyToJoin` consequently emits only the value equality and leaves `o.k`
inside the right subtree, so final child-slot validation rejects the query. The
new Window/Repeat/Limit/Generate guards do not cover this multi-child shape.
Please either reject Join on the path to the outer-reading Filter or extract
the predicate from the branch, and add a planner oracle for this query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
