Omega359 commented on issue #7698: URL: https://github.com/apache/datafusion/issues/7698#issuecomment-2711715994
After spending a long time looking at different parts of the planning code I've come to the following conclusions: 1. It's complex 2. It's sequential with no obvious avenue to make parallel 3. It seems to spend a lot of time in EquivalenceClass doing hashing Unless I get an epiphany somehow I'm leaning towards letting others tackle various solutions to planning performance (for example, https://github.com/apache/datafusion/issues/15045) and taking a different approach locally which is attempting to cache and reuse ExecutionPlan's rather than regenerating them over and over again. I don't know how feasible that is but I guess I'll see. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
