huginn created HBASE-30392:
------------------------------
Summary: Exclude checksum bytes from cache-on-write HFile block
data size
Key: HBASE-30392
URL: https://issues.apache.org/jira/browse/HBASE-30392
Project: HBase
Issue Type: Bug
Components: BlockCache
Affects Versions: 2.4.11
Reporter: huginn
h3. What happens
When an HFile block is cached on write, the cache buffer contains only the
block header and on-disk data. The HFileBlock metadata field
onDiskDataSizeWithHeader is nevertheless populated with a value that also
includes trailing checksum bytes.
h3. When it happens
This occurs when cache-on-write creates a block through
HFileBlock.Writer#getBlockForCaching, including compressed cache and index or
inline blocks.
h3. Impact
The metadata and buffer length disagree. Subsequent cache block validation or
unpacking can use an incorrect data length, causing cache equality failures or
read or decompression errors.
h3. Root cause
HFileBlock.Writer#getBlockForCaching uses onDiskBlockBytesWithHeader.size() +
onDiskChecksum.length even though cloneOnDiskBufferWithHeader() copies only the
header and data, and the cache context has no checksum bytes. The invariant for
onDiskDataSizeWithHeader is the header plus data length, excluding checksums.
h3. Proposed fix
Set onDiskDataSizeWithHeader to onDiskBlockBytesWithHeader.size(), while
preserving onDiskSizeWithoutHeader, which continues to include checksum bytes
for the physical file size.
h3. Reproduction / Testing
Add test evidence here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
