blaginin commented on issue #14563: URL: https://github.com/apache/datafusion/issues/14563#issuecomment-2652180854
Okay, so I think the issue is that with every `.with_column_renamed` / `.with_column` we add a new projection - that creates a lot of layers and each time adding a new one is more and more complicated because we need to revisit all existing ones: <img width="1642" alt="Image" src="https://github.com/user-attachments/assets/12715024-48dc-490c-8a2b-21c01ab49f10"; /> I feel like a good start would be to reuse the existing projection if it's already on the top. It won't cover all cases but cover the majority (including the one in the benches). It can be something like this: https://github.com/apache/datafusion/compare/main...blaginin:datafusion:wip-reuse-projection?expand=1. I'll finish the code if that makes sense to you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
