[
https://issues.apache.org/jira/browse/CASSANDRA-21550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18102215#comment-18102215
]
Chris Lohfink edited comment on CASSANDRA-21550 at 8/5/26 8:37 PM:
-------------------------------------------------------------------
*Benchmark results*
Splitting one SSTable 8 ways, cold cache. Comparison is against
{{{}SSTableSplitter{}}}, the full
deserialise/reserialise path that {{sstablesplit}} runs today.
||Configuration||Wall clock||CPU time||Read from device||Written to
device||Heap churned||
|Existing|257.9 s|148.5 s|62.65 GiB|62.64 GiB|69.6 GiB|
|Zero-copy, byte copy (no reflink)|258.6 s|72.6 s|62.67 GiB|54.85 GiB|257 MiB|
|Zero-copy + reflink|123.2 s|46.4 s|62.66 GiB|57 MiB|267 MiB|
|*Zero-copy + reflink, no digest*|*0.77 s*|*0.93 s*|*24.6 MiB*|*57 MiB*|*257
MiB*|
Against the existing rewrite, the configuration we would run (reflink on,
digest off) is
{*}335× faster{*}, uses {*}160× less CPU{*}, writes {*}1,125× fewer bytes{*},
and allocates {*}277× less heap{*}.
{panel:title=Test setup}
* *Host* — r5d.2xlarge, 8 vCPU, 62 GiB RAM, local NVMe, xfs formatted {{-m
reflink=1}}, JDK 21.
* *Workload* — 63 GiB parent (64,125 MiB, 1,048,576 partitions, 64 KiB
partitions of 4 KiB rows,
LZ4, 16 KiB chunks), split 8 ways. Figures above are the 63 GiB point of a 1
GiB → 63 GiB sweep,
84 measured splits, 3 iterations per point, medians.
* *Cold cache* — every component evicted with {{posix_fadvise(DONTNEED)}}
before each timed run, and
checked rather than assumed: {{iostat}} recorded 1,124 GiB read at the device
over the run against
1,125 GiB predicted from {{/proc/self/io}}. Nothing was served from page
cache.
* *Baseline is real* — a genuine {{SSTableSplitter.SplittingCompactionTask}}
over a real
{{ColumnFamilyStore}} and {{LifecycleTransaction}}, not a reimplementation.
It runs at {{NO_GC}} and
purges no tombstones, so 148.5 s of CPU is the *floor* for the rewrite
approach, not an inflated
number. {{compaction_throughput}} pinned to 0, otherwise this would measure
the throttle.
* *Correctness* — every run verified that the children carry exactly the
parent's partition keys
(count plus XOR and sum of a 64-bit hash per key) before its timing was kept.
Zero failures in 84 runs.
{panel}
was (Author: clohfink):
*Benchmark results*
Splitting one SSTable 8 ways, cold cache. Comparison is against
{{{}SSTableSplitter{}}}, the full
deserialise/reserialise path that {{sstablesplit}} runs today.
||Configuration||Wall clock||CPU time||Read from device||Written to
device||Heap churned||
|Existing|257.9 s|148.5 s|62.65 GiB|62.64 GiB|69.6 GiB|
|Zero-copy, byte copy (no reflink)|258.6 s|72.6 s|62.67 GiB|54.85 GiB|257 MiB|
|Zero-copy + reflink|123.2 s|46.4 s|62.66 GiB|57 MiB|267 MiB|
|*Zero-copy + reflink, no digest*|*0.77 s*|*0.93 s*|*24.6 MiB*|*57 MiB*|*257
MiB*|
Against the existing rewrite, the configuration we would run (reflink on,
digest off) is
{*}335× faster{*}, uses {*}160× less CPU{*}, writes {*}1,125× fewer bytes{*},
and allocates {*}277× less heap{*}.
> Zero-copy sstable splitting for anticompaction, and partial-sstable zero-copy
> streaming
> ---------------------------------------------------------------------------------------
>
> Key: CASSANDRA-21550
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21550
> Project: Apache Cassandra
> Issue Type: New Feature
> Reporter: Chris Lohfink
> Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Anticompaction and range streaming both spend most of their cost re-doing
> work whose result they already have on disk. Anticompaction rewrites every
> row of an sstable through three writers only to place each partition into a
> repaired, transient or unrepaired child. Streaming a subset of an sstable's
> ranges takes the row-by-row path, where the sender is cheap (whole
> compression chunks are sent verbatim) but the receiver decompresses,
> deserialises, re-serialises and recompresses every row and then rebuilds the
> index, filter and summary it could have been handed. Both are avoidable when
> the bytes wanted are a contiguous run of the parent's compression chunks: the
> chunks can be copied (or extent-shared) verbatim and every other component
> rebuilt from an index-only pass, with no row deserialised on either side.
> This adds that machinery and the two callers for it. Both are off by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
