zhuqi-lucas commented on PR #16690: URL: https://github.com/apache/datafusion/pull/16690#issuecomment-3054819187
> I wonder if we should focus on more parallelisation in `SortPreservingMerge` (besides improving the hot paths like `gc` `interleave`, comparisons, etc.) > > I think there might be two areas: > > * Doing (some of the) merge algorithm itself in parallel - I am not sure what would the best way forward here, but it seems it could give the largest gains, as merging is currently done in a single thread. > * Doing conversion to `Row` format in parallel - it seems currently this is done in a single thread, it should give some (smaller) speedup Thank you @Dandandan , interesting idea, i agree if we can improve this point, it will help not only stringview both also other type, so it will benefit more cases, i will investigate! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
