Surya Hebbar has uploaded a new patch set (#2). (
http://gerrit.cloudera.org:8080/22718 )
Change subject: IMPALA-13923: Support more compression levels for ZSTD and ZLIB
......................................................................
IMPALA-13923: Support more compression levels for ZSTD and ZLIB
This patch adds support for more compression levels for ZLIB and ZSTD,
and expands the negative range of ZSTD compression levels.
This is done by refactoring the implementation and tests,
to make compression levels an optional parameter in CodecInfo.
Hence, this also solves IMPALA-10630.
Added support for parsing additional compression levels for ZSTD
and ZLIB in "compression_codec" query option.
The following additional compression levels are now supported.
For ZSTD,
ZSTD_minCLevel(-ZSTD_TARGETLENGTH_MAX) to ZSTD_maxCLevel(20)
For ZLIB i.e. ZLIB, GZIP and DEFLATE,
Z_DEFAULT_COMPRESSION(-1) to Z_BEST_COMPRESSION(9)
Added tests for
* Additional compression levels for GZIP
- from Z_BEST_SPEED, Z_BEST_COMPRESSION
* Negative compression levels for ZSTD
- Lowered the range to ZSTD_minCLevel()
- Seperated tests for negative compression levels
* Query option - "compression_codec" for the newly added formats
and compression levels
Change-Id: I5b98c735246f08e04598a4e752c8cca04e31a88a
---
M be/src/runtime/tmp-file-mgr.cc
M be/src/runtime/tmp-file-mgr.h
M be/src/service/query-options-test.cc
M be/src/service/query-options.cc
M be/src/util/codec.cc
M be/src/util/codec.h
M be/src/util/compress.cc
M be/src/util/compress.h
M be/src/util/decompress-test.cc
M be/src/util/parse-util.cc
M be/src/util/parse-util.h
11 files changed, 133 insertions(+), 54 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/22718/2
--
To view, visit http://gerrit.cloudera.org:8080/22718
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5b98c735246f08e04598a4e752c8cca04e31a88a
Gerrit-Change-Number: 22718
Gerrit-PatchSet: 2
Gerrit-Owner: Surya Hebbar <sheb...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sheb...@cloudera.com>
