[
https://issues.apache.org/jira/browse/KUDU-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daojiang Yan updated KUDU-3779:
-------------------------------
Attachment: KUDU-3779-bench-logs.tar.gz
> Support encrypted metadata in the log block container in-memory replay path
> ---------------------------------------------------------------------------
>
> Key: KUDU-3779
> URL: https://issues.apache.org/jira/browse/KUDU-3779
> Project: Kudu
> Issue Type: Improvement
> Reporter: Daojiang Yan
> Assignee: Daojiang Yan
> Priority: Minor
> Attachments: KUDU-3779-bench-logs.tar.gz
>
>
> KUDU-3778 change introducing
> --log_container_metadata_inmem_replay_threshold_bytes) added an in-memory
> replay fast path for LogBlockContainerNativeMeta::ProcessRecords(): instead
> of issuing two preadv() syscalls per record, the entire metadata file is
> slurped into memory once and ReadablePBContainerFile then reads from a
> MemoryReadableFile wrapper. This drastically reduces syscalls during the
> "Reading filesystem" stage of tserver startup on hosts with many containers.
> The current implementation intentionally falls back to the streaming path
> whenever the metadata file is encrypted (i.e.
> RandomAccessFile::GetEncryptionHeaderSize() > 0). As a result, encrypted
> clusters do not benefit from the optimization.
> *Why encrypted metadata is excluded today*
> ReadablePBContainerFile recovers from an incomplete trailing record (typical
> after a crash mid-write, when the on-disk tail is a file-hole filled with
> zeros) by relying on a per-slice optimization inside DoDecryptV(): it detects
> all-zero ciphertext slices and leaves them as zeros in plaintext rather than
> running them through AES. The PB container parser then sees a zeroed tail and
> returns Status::Incomplete, which the caller handles by truncating the
> partial record.
> If we preload the whole payload as a single buffer and serve all subsequent
> reads from memory, the decryption layer sees one bulk slice instead of
> per-record slices. The trailing zero region is no longer aligned with a slice
> boundary, the zero-slice shortcut is defeated, and the file-hole tail
> decrypts to garbled plaintext. The PB parser would then misclassify a benign
> crash-truncation as corruption.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
