andygrove commented on PR #1192: URL: https://github.com/apache/datafusion-comet/pull/1192#issuecomment-2564419774
> Hmm, the difference between ZSTD and LZ4 looks not significant? Does the benchmark of TPC-H shown in the description include native shuffle block decompression and decoding? Do we have the number of difference between current Comet (ZSTD) and this patch (ZSTD + native decompression and decoding)? Sure, here are fresh benchmarks | Branch | Compression Codec | TPC-H Time | |-|-|-| | 0.4.0 | ZSTD | 364 | | main | ZSTD | 364 | | native-decode | ZSTD | 367 | | native-decode | Snappy 343 | | native-decode | LZ4 | 340 | I had hoped for a bigger win, but 340 compared to 364 is still a 7% improvement overall. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
