Dandandan opened a new issue, #16431: URL: https://github.com/apache/datafusion/issues/16431
### Is your feature request related to a problem or challenge? I ran some benchmarks in DataFusion (sort_tpch) and I saw that `interleave_views` take up a large amount of time for the sorting benchmark (sort_tpch). <img width="1234" alt="Image" src="https://github.com/user-attachments/assets/00dd8025-b331-446a-86b4-48e20308e700"; /> It shows up taking roughly 17% of the samples of SortPreservingMergeExec (of 77%, so it's about 25% of the samples). Looking at the samples, it shows that a lot of time is spent managing a hashmap, rehashing, allocating, etc. <img width="785" alt="Image" src="https://github.com/user-attachments/assets/7727a930-eb84-4ffa-8311-a3269650ea06"; /> ### Describe the solution you'd like We should be able to optimize this. I am not 100% sure what the purpose of the hashmap is here, but we should be able to optimize this to a great extent. I think we can combine it with the improvements that are done to `concat` and `coalesce` @alamb ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
