XiDuo You created SPARK-59492:
---------------------------------
Summary: Remove the local sort feeding a redundant partial
WindowGroupLimit
Key: SPARK-59492
URL: https://issues.apache.org/jira/browse/SPARK-59492
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: XiDuo You
RemoveRedundantWindowGroupLimits only reaches a partial WindowGroupLimitExec
that sits directly below the final one. When EnsureRequirements puts a local
sort between the final node and a GroupPartitionsExec -- the sort that gives
the final node its ordering -- the partial node below the grouping is kept
although removing it cannot change the result.
Look through that local sort, and remove the local sort that fed the partial
node with it: a sort between the two nodes already orders the rows the final
node ranks as a whole, which makes the one below dead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
