[
https://issues.apache.org/jira/browse/CASSANDRA-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114577#comment-18114577
]
Dmitry Konstantinov edited comment on CASSANDRA-21637 at 9/12/26 12:00 PM:
---------------------------------------------------------------------------
I cleared the test data, so I generated new one and run before vs after x chunk
cache enabled (with default size)/disabled, so the absolute numbers can be a
bit different.
{code:java}
Before → After, file cache DISABLED
-----------------------------------------------------------------------
metric Before After delta
-----------------------------------------------------------------------
Op rate 344,966 op/s 366,330 op/s +6.2%
Partition rate 334,423 pk/s 355,133 pk/s +6.2%
Row rate 3,344,232 row/s 3,551,332 row/s +6.2%
Latency mean 1.4 ms 1.3 ms -7.1%
Latency median 1.2 ms 1.2 ms +0.0%
Latency 95th percentile 2.9 ms 2.7 ms -6.9%
Latency 99th percentile 8.1 ms 5.8 ms -28.4%
Total partitions 14,541,576 14,541,536
Total GC count 13 12 -7.7%
Total GC memory 239.890 GiB 221.436 GiB -7.7%
CPU per million ops 220.1 202.9 -7.9%
Before → After, file cache ENABLED
-----------------------------------------------------------------------
metric Before After delta
-----------------------------------------------------------------------
Op rate 340,201 op/s 339,252 op/s -0.3%
Partition rate 329,808 pk/s 328,896 pk/s -0.3%
Row rate 3,298,083 row/s 3,288,961 row/s -0.3%
Latency mean 1.4 ms 1.4 ms +0.0%
Latency median 0.9 ms 0.9 ms +0.0%
Latency 95th percentile 3.3 ms 3.2 ms -3.0%
Latency 99th percentile 12.8 ms 12.1 ms -5.5%
Total partitions 14,541,773 14,542,110
Total GC count 15 12 -20.0%
Total GC memory 227.559 GiB 221.225 GiB -2.8%
CPU per million ops 226.2 225.4 -0.3%
The 2×2
┌─────────────┬───────────────┬────────────────┬──────────────┐
│ op/s │ cache enabled │ cache disabled │ cache effect │
├─────────────┼───────────────┼────────────────┼──────────────┤
│ Before │ 340,201 │ 344,966 │ +1.4% │
├─────────────┼───────────────┼────────────────┼──────────────┤
│ After │ 339,252 │ 366,330 │ +8.0% │
├─────────────┼───────────────┼────────────────┼──────────────┤
│ code effect │ −0.3% │ +6.2% │ │
└─────────────┴───────────────┴────────────────┴──────────────┘
{code}
was (Author: dnk):
I cleared the test data, so I generated new one and run before vs after x chunk
cache enabled (with default size)/disabled, so the absolute numbers can be a
bit different.
{code}
Before → After, file cache DISABLED
-----------------------------------------------------------------------
metric Before After delta
-----------------------------------------------------------------------
Op rate 344,966 op/s 366,330 op/s +6.2%
Partition rate 334,423 pk/s 355,133 pk/s +6.2%
Row rate 3,344,232 row/s 3,551,332 row/s +6.2%
Latency mean 1.4 ms 1.3 ms -7.1%
Latency median 1.2 ms 1.2 ms +0.0%
Latency 95th percentile 2.9 ms 2.7 ms -6.9%
Latency 99th percentile 8.1 ms 5.8 ms -28.4%
Total partitions 14,541,576 14,541,536
Total GC count 13 12 -7.7%
Total GC memory 239.890 GiB 221.436 GiB -7.7%
CPU per million ops 220.1 202.9 -7.9%
Before → After, file cache ENABLED
-----------------------------------------------------------------------
metric Before After delta
-----------------------------------------------------------------------
Op rate 340,201 op/s 339,252 op/s -0.3%
Partition rate 329,808 pk/s 328,896 pk/s -0.3%
Row rate 3,298,083 row/s 3,288,961 row/s -0.3%
Latency mean 1.4 ms 1.4 ms +0.0%
Latency median 0.9 ms 0.9 ms +0.0%
Latency 95th percentile 3.3 ms 3.2 ms -3.0%
Latency 99th percentile 12.8 ms 12.1 ms -5.5%
Total partitions 14,541,773 14,542,110
Total GC count 15 12 -20.0%
Total GC memory 227.559 GiB 221.225 GiB -2.8%
CPU per million ops 226.2 225.4 -0.3%
The 2×2
┌─────────────┬───────────────┬────────────────┬──────────────┐
│ op/s │ cache enabled │ cache disabled │ cache effect │
├─────────────┼───────────────┼────────────────┼──────────────┤
│ Before │ 340,201 │ 344,966 │ +1.4% │
├─────────────┼───────────────┼────────────────┼──────────────┤
│ After │ 339,252 │ 366,330 │ +8.0% │
├─────────────┼───────────────┼────────────────┼──────────────┤
│ code effect │ −0.3% │ +6.2% │ │
└─────────────┴───────────────┴────────────────┴──────────────┘
{code}
Flamegraphs:
[^noser_base_btiNoDoubleReadBefore_fileCacheEnabled_wall.html]
[^noser_base_btiNoDoubleReadBefore_fileCacheEnabled_cpu.html]
[^noser_base_btiNoDoubleReadBefore_fileCacheDisabled_wall.html]
[^noser_base_btiNoDoubleReadBefore_fileCacheDisabled_cpu.html]
[^noser_base_btiNoDoubleReadAfter_fileCacheEnabled_wall.html]
[^noser_base_btiNoDoubleReadAfter_fileCacheEnabled_cpu.html]
[^noser_base_btiNoDoubleReadAfter_fileCacheDisabled_wall.html]
[^noser_base_btiNoDoubleReadAfter_fileCacheDisabled_cpu.html]
The issue with chunk cache: if the eviction rate is high then we have a
contention on a common lock on Caffeine cache. I use uniform distribution in my
test, so it is a kind of worst case probably for the cache
> Avoid double chunk read for BTI small partitions during a select query
> execution
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-21637
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21637
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Local/SSTable
> Reporter: Dmitry Konstantinov
> Assignee: Dmitry Konstantinov
> Priority: Normal
> Fix For: 6.0.x, 7.x
>
> Attachments: image-2026-09-02-15-59-03-586.png,
> image-2026-09-06-17-35-44-923.png, noser_base_btiNoDoubleReadAfter_cpu.html,
> noser_base_btiNoDoubleReadAfter_fileCacheDisabled_cpu.html,
> noser_base_btiNoDoubleReadAfter_fileCacheDisabled_wall.html,
> noser_base_btiNoDoubleReadAfter_fileCacheEnabled_cpu.html,
> noser_base_btiNoDoubleReadAfter_fileCacheEnabled_wall.html,
> noser_base_btiNoDoubleReadBefore_cpu.html,
> noser_base_btiNoDoubleReadBefore_fileCacheDisabled_cpu.html,
> noser_base_btiNoDoubleReadBefore_fileCacheDisabled_wall.html,
> noser_base_btiNoDoubleReadBefore_fileCacheEnabled_cpu.html,
> noser_base_btiNoDoubleReadBefore_fileCacheEnabled_wall.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> BtiTableReader.getExactPosition() opens a data-file reader to verify the
> partition key when the trie payload is negative (means the partition small
> enough to have no row index entry). It then closed that reader, and
> AbstractSSTableIterator immediately reopened the same file at the same
> position. So, we read the same chunk twice in such scenario as well as run
> LZ4 decompression and CRC32 check twice. We can re-use the reader to avoid it.
> !image-2026-09-02-15-59-03-586.png|width=600!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
