adriangb commented on PR #15770: URL: https://github.com/apache/datafusion/pull/15770#issuecomment-2971391046
Thanks for the review @alamb! I'll try to summarize the high level issues: 1. Mutli-column order by not working. This seems like a bug / oversight. Fundamentally I don't see any reason it shouldn't work, I'll have to investigate why. 2. Interaction with `EnforceSorting` & co. After grokking the code for a while I'm pretty convinced this is the only way to do things correctly. `SortExec` needs to establish a link (via references) with the `DataSourceExec`, and that needs to happen _before_ `EnforceSorting` runs because it might modify the plan in the process. So we have no choice but to teach `EnforceSorting` how to preserve this link. But that's the part I'm not sure I did correctly since I don't fully understand how `EnforceSorting` works. There may be other alternatives here. @berkaysynnada suggested that the right rule ordering might "just work", but neither of us could get it to. We might be able to split the filter pushdown into two steps: static (cannot make assumptions about reference links but can modify the plan tree, e.g. for `FitlerExec`) and dynamic (can make reference links but cannot modify the plan tree, e.g. `SortExec` and `HashJoin`). 3. Formatting in explain plans. I'm open to suggestions on this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
