alamb commented on issue #19826: URL: https://github.com/apache/datafusion/issues/19826#issuecomment-3759739443
> For 4xlarge instances, almost all cases are improved, but I observed some regression in small instances (2x or x), like q3 q4 q5 now needs 2~3 seconds to run ([here](https://github.com/ClickHouse/ClickBench/pull/749/changes#diff-6887ea4cb78ed75a19622030189b040882bab594e1ad5587052dcc46e67919e5R16-R18)). @waynexia I suspect this is a result of using more memory somehow (and this swapping more). What I think need to happen is that DataFusion's "out of core" story needs to improve (aka we shouldn't need swap space, we should be using the memory manager and spill mechanism). However, when I last tried this I encountered issues running some of the queries - https://github.com/apache/datafusion/issues/19425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
