GitHub user pepijnve added a comment to the discussion: Multiple 'group by's, one scan
I read through the linked issues in the meantime. I think what we're trying to do is closest to the Splitter idea described in the linked document at https://github.com/apache/datafusion/pull/8558#issuecomment-1862968970 The part we've been struggling with a bit is how to fit this in the framework of DataFusion without breaking assumptions. While it's definitely already feasible to create diamond plans in code, my colleague and I got the impression that most code has been written under the assumption that nodes only have a single parent. One example that comes to mind was projection pushdown where each branch would push down only the columns it needed. If you have any guidance/pointers on how to go about tackling this problem without going against the grain of the library, that would be much appreciated. Happy to contribute the work upstream if we can get beyond the point of a hack to something sufficiently general and usable for others. GitHub link: https://github.com/apache/datafusion/discussions/15982#discussioncomment-13160947 ---- This is an automatically sent email for github@datafusion.apache.org. To unsubscribe, please send an email to: github-unsubscr...@datafusion.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
