lorenzhs opened a new issue, #51308: URL: https://github.com/apache/arrow/issues/51308
### Describe the bug, including details regarding any error messages, version, and platform. https://github.com/apache/arrow/blob/cd304b22da0ca1dc92edf86e6476d733f5ea5b05/cpp/src/arrow/util/compression_zlib.cc#L497-L501 This is wasteful: `InitCompressor` on line 497 calls `deflateInit2`, which at `windowBits=15` and `memLevel=8` allocates roughly 256 KB. `InitDecompressor` on line 501 calls `EndCompressor` which does `deflateEnd` and frees it again. It's pure waste. The only thing it gives us is some validation on the compression level. The current code leaves the `GZipCodec` with a valid decompressor (which also allocates, but less) which is fine for the case where the caller wants to use the codec to do decompression, but is extra-wasteful for the case where the caller wants to compress: it allocates a compressor, frees it, allocates a decompressor, frees it, then allocates a compressor again. I think the correct fix is https://github.com/firebolt-db/arrow/pull/47, I'm happy to file it as a PR here as well. ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
