[
https://issues.apache.org/jira/browse/CASSANDRA-21683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116521#comment-18116521
]
Jon Haddad commented on CASSANDRA-21683:
----------------------------------------
The builder now encodes each partition key's byte-comparable form once, into a
reusable scratch buffer, and reuses those bytes for every step that used to
re-encode the key.
The changes in \{{PartitionIndexBuilder}}:
- \{{materialize()}} writes a key's comparable bytes into a scratch buffer and
returns the length. The buffer grows on demand and is reused across partitions,
so in steady state this path allocates nothing.
- \{{diffPoint}} is now an array compare (\{{Arrays.mismatch}} over the two
buffers, plus one to match the old convention) instead of encoding the keys
again.
- The prefix handed to the trie is a \{{ByteComparable.fixedLength}} view over
the captured bytes, not a freshly cut byte source.
- A three-slot ring of buffers holds the three keys that are live at the same
instant: the current key, the previous key, and the previous key the trie reads
again when the next key is added. The ring advances one slot per key. Two
buffers would overwrite the previous key before the trie was done with it.
Correctness
- \{{fixedLength}} does not stop at the end of a key's bytes the way the old
\{{ByteComparable.cut}} did, so the code limits the prefix length explicitly to
build the exact same prefix.
- An assertion guards the ring so a slot is never reused while it still holds
the last key or the key the trie reads again.
- The on-disk index is byte-identical to before.
Verification
- \{{PartitionIndexBuilderReencodeTest}} runs the patched builder with a
recording trie writer, then asserts every prefix and payload matches the
original \{{diffPoint}}/\{{cut}} logic byte for byte. It also checks the
previous-key re-read, so a wrong ring size (three slots down to two) fails the
test. It covers adversarial byte shapes and a non-default partitioner.
- JMH \{{PartitionIndexBuildBench}} with \{{-prof gc}}: allocation per
partition drops about 288 bytes, -59.3% at one million keys, with no change to
build time.
The builder now encodes each partition key's byte-comparable form once, into a
reusable scratch buffer, and reuses those bytes for every step that used to
re-encode the key.
The changes in \{{PartitionIndexBuilder}}:
- \{{materialize()}} writes a key's comparable bytes into a scratch buffer and
returns the length. The buffer grows on demand and is reused across partitions,
so in steady state this path allocates nothing.
- \{{diffPoint}} is now an array compare (\{{Arrays.mismatch}} over the two
buffers, plus one to match the old convention) instead of encoding the keys
again.
- The prefix handed to the trie is a \{{ByteComparable.fixedLength}} view over
the captured bytes, not a freshly cut byte source.
- A three-slot ring of buffers holds the three keys that are live at the same
instant: the current key, the previous key, and the previous key the trie reads
again when the next key is added. The ring advances one slot per key. Two
buffers would overwrite the previous key before the trie was done with it.
Correctness
- \{{fixedLength}} does not stop at the end of a key's bytes the way the old
\{{ByteComparable.cut}} did, so the code limits the prefix length explicitly to
build the exact same prefix.
- An assertion guards the ring so a slot is never reused while it still holds
the last key or the key the trie reads again.
- The on-disk index is byte-identical to before.
Verification
- \{{PartitionIndexBuilderReencodeTest}} runs the patched builder with a
recording trie writer, then asserts every prefix and payload matches the
original \{{diffPoint}}/\{{cut}} logic byte for byte. It also checks the
previous-key re-read, so a wrong ring size (three slots down to two) fails the
test. It covers adversarial byte shapes and a non-default partitioner.
- JMH \{{PartitionIndexBuildBench}} with \{{-prof gc}}: allocation per
partition drops about 288 bytes, -59.3% at one million keys, with no change to
build time.
> Reduce allocations in BTI writes
> --------------------------------
>
> Key: CASSANDRA-21683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21683
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Local/SSTable
> Reporter: Jon Haddad
> Assignee: Jon Haddad
> Priority: Normal
>
> When Cassandra writes a BTI SSTable, it builds a partition index: a trie that
> maps each partition key to its position on disk. To place a key in the trie,
> the builder first converts it to a byte-comparable form, a byte string that
> sorts in the same order as the key.
> That conversion runs repeatedly. For each partition, the builder re-encodes
> the same one or two keys about four times: once to find where a key differs
> from its neighbour, once to cut the shared prefix, and again when the trie
> re-reads the previous key. Each encode allocates a chain of throwaway objects
> (ByteSource, Multi, varargs arrays, escapers), roughly nineteen short-lived
> objects per partition. Across a flush or compaction with millions of
> partitions, this is a large amount of garbage, and it pressures the garbage
> collector
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
