cloud-fan commented on code in PR #49955:
URL: https://github.com/apache/spark/pull/49955#discussion_r1959745875
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala:
##########
@@ -848,6 +848,15 @@ object LimitPushDown extends Rule[LogicalPlan] {
case LocalLimit(exp, u: Union) =>
LocalLimit(exp, u.copy(children =
u.children.map(maybePushLocalLimit(exp, _))))
+ // If limit node is present, we should propagate it down to UnionLoop, so
that it is later
+ // propagated to UnionLoopExec.
+ // Limit node is constructed by placing GlobalLimit over LocalLimit (look
at Limit apply method)
+ // that is the reason why we match it this way.
+ case g @ GlobalLimit(IntegerLiteral(limit), l @ LocalLimit(_, p @
Project(_, ul: UnionLoop))) =>
Review Comment:
nit: `case Limit(...)` matches a `GlobalLimit` wrapping a `LocalLimit` with
the same value, we should use it instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
