alamb commented on issue #15323: URL: https://github.com/apache/datafusion/issues/15323#issuecomment-2776820608
> I think I have the the same problem but in `AggregateExec` when using `row_hash`, as it spills as well and use `SortPreservingMergeStream`. > > I think the solution should actually be in `SortPreservingMergeStream` rather than `SpillFileManager` no? although it does not spawn blocking threads it should support the multiple levels to merge I am not sure / familiar enough with the code to know off the top of my head. I do think having hash and sort use the same codepath (that we can then go optimize a lot) sounds like a great idea -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
