Dandandan commented on issue #20902: URL: https://github.com/apache/datafusion/issues/20902#issuecomment-4059947271
@nathanb9 Currently this is just me running some experiments and seeing what helps or gives non measurable improvement So far I tried the following (besides morsels): * Pinning threads - currently can't see a big diff in benchmarks. Could perhaps change with the morsel approach (as it has higher cpu usage), but perhaps OS scheduling does do the scheduling better or bandwidth has improved much since (and decompression / decoding perf hasn't) that it doesn't have any effect. * Blocking decode (i.e. local to thread) vs `spawn_blocking` the same, only minor changes. * Some prefetching IO: seems to help _sometimes_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
