comphead commented on PR #7256: URL: https://github.com/apache/datafusion-comet/pull/7256#issuecomment-6512606649
Thanks @sunchao, both alias cases are fixed in dc94ccf3d3. `with_declared_order` used to return early whenever its input declared an order. Now it returns early only when the declared order already satisfies the one Spark sent. Otherwise, if `keeps_source_order` trusts the input, it adds the order next to what the input declares. `keeps_source_order` walks through both window operators and `SortedInputExec`, since all of them keep their input's order. In your first example the upper window's input declares `a`, and `b` is now added. This also extends a partial declaration at the upper window, your earlier suggestion. So a suffix key the lower window leaves out is no longer the only way for the upper window to get its order. An operator that may move rows, such as a native sort, still blocks the declaration. `declared_order_extends_an_order_the_input_declares` in `planner.rs` covers the three outcomes: an order the input already satisfies is left alone, a trusted input gets the second order, and a sort gets nothing. The first query in `stacked native windows over a sorted cache with aliases of a sort key` is your 12-row example, extended to 24 rows with NULL keys: `k AS a, k AS b` over a cache sorted by `k`, then `sum(v) OVER (PARTITION BY a)` under `sum(s) OVER (PARTITION BY b)`. It checks the rows, that the lower window sends `a` and the upper `b`, and that no sort is planned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
