Marc Beyerle created CASSANDRA-21514:
----------------------------------------
Summary: DeflateCompressor calculates compressed buffer size too
small when HW-accelerated deflate (DFLTCC) is used on s390x, causing
CorruptSSTableException
Key: CASSANDRA-21514
URL: https://issues.apache.org/jira/browse/CASSANDRA-21514
Project: Apache Cassandra
Issue Type: Bug
Components: Feature/Compression
Reporter: Marc Beyerle
Attachments: patch.txt
h2. *Background*
IBM Z mainframes (architecture: s390x) provide a hardware-accelerated deflate
instruction called *{{DFLTCC}}* (Deflate Conversion Call). {{DFLTCC}} is
transparently exploited by the IBM Semeru / OpenJ9 JVM whenever a Java
application uses {{java.util.zip.Deflater}} or {{java.util.zip.Inflater}} - no
code changes are required by the application developer. In practice, {{DFLTCC}}
accelerates deflate-based compression by *10×* to *100×* compared to a pure
software implementation, making {{DeflateCompressor}} a highly attractive
choice on this platform.
h2. *The bug*
The current implementation of {{initialCompressedBufferLength(int sourceLen)}}
in {{DeflateCompressor}} computes the maximum compressed output size using a
manual formula that mirrors the classic zlib {{deflateBound()}} calculation for
a pure software deflate stream. However, {{DFLTCC}} generates a slightly
different bit-stream layout: it requires additional bits for block headers,
end-of-block markers, and padding that are specific to the hardware
implementation. As a result, the computed maximum buffer size can be *too
small* to hold the actual compressed output produced by {{{}DFLTCC{}}}.
When the compressed data exceeds the allocated buffer, the excess bytes are
silently discarded. On the read path, Cassandra then detects the data
inconsistency and throws a {{{}CorruptSSTableException{}}}, making the affected
SSTables unreadable.
The s390x-specific deflate buffer overhead is documented in the zlib fork that
provides {{DFLTCC}} support - see the {{DEFLATE_BOUND_COMPLEN}} macro in
[contrib/s390/dfltcc.h|https://github.com/iii-i/zlib/blob/dfltcc/contrib/s390/dfltcc.h]
for the precise accounting.
h2. *Proposed fix*
Instead of maintaining a manual buffer-size formula in
{{{}initialCompressedBufferLength(){}}}, delegate to the native zlib library's
own {{deflateBound()}} function. This has two benefits:
* {*}Correctness on s390x{*}: The zlib library shipped with
{{{}DFLTCC{}}}-aware systems already accounts for the additional overhead in
its own {{{}deflateBound(){}}}, so the buffer will always be sized correctly
regardless of the underlying implementation.
* {*}Future-proofing on all platforms{*}: Any future change to zlib's internal
compression format or bound calculation is automatically picked up, without
requiring a corresponding change in Cassandra.
The proposed change calls {{deflateBound()}} via JNA. Because
{{initialCompressedBufferLength()}} is called infrequently, the JNI transition
overhead is negligible. Benchmark results on s390x confirm {*}no measurable
performance regression{*}.
As a safe fallback, if the native zlib library is not available on the host OS
image, the implementation gracefully falls back to the existing manual
calculation, preserving full backward compatibility.
h2. *Steps to reproduce*
* Run Apache Cassandra 5.0.8 on an IBM Z host (s390x) with a DFLTCC-capable
JVM (IBM Semeru / OpenJ9).
* Configure {{DeflateCompressor}} as the SSTable compressor for a table.
* Write a sufficient volume of data to trigger compression of chunks whose
compressed output exceeds the currently calculated buffer size.
* Attempt to read back the written data.
{*}Expected result{*}: Data is read back correctly.
{*}Actual result{*}: {{CorruptSSTableException}} is thrown.
h2. *Impact & risk assessment*
* {*}Affected platforms{*}: s390x with {{{}DFLTCC{}}}-capable JVM. All other
platforms are unaffected by the bug but benefit from the future-proofing aspect
of the fix.
* {*}Severity{*}: Data read failures ({{{}CorruptSSTableException{}}}) for any
table using {{DeflateCompressor}} on the affected platform.
* {*}Fix risk{*}: Low. The change is confined to a single method in
{{{}DeflateCompressor{}}}, the fallback preserves existing behaviour, and there
is no performance impact on the hot path.
{*}Note{*}: Attached my proposed patch to this ticket as an attachment.
Successfully tested and verified it in my lab environment. Happy to provide
additional details and looking forward to any review feedback!
Marc
[^patch.txt]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
