[
https://issues.apache.org/jira/browse/CASSANDRA-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18046432#comment-18046432
]
Dmitry Konstantinov commented on CASSANDRA-21083:
-------------------------------------------------
An initial draft:
https://github.com/apache/cassandra/compare/trunk...netudima:cassandra:CASSANDRA-21083-trunk?expand=1
The current results (same test as here:
https://issues.apache.org/jira/browse/CASSANDRA-20226):
{code:java}
Results:
Op rate : 192,197 op/s [insert: 192,197 op/s]
Partition rate : 192,197 pk/s [insert: 192,197 pk/s]
Row rate : 1,921,965 row/s [insert: 1,921,965 row/s]
Latency mean : 1.5 ms [insert: 1.5 ms]
Latency median : 1.1 ms [insert: 1.1 ms]
Latency 95th percentile : 2.4 ms [insert: 2.4 ms]
Latency 99th percentile : 7.1 ms [insert: 7.1 ms]
Latency 99.9th percentile : 24.8 ms [insert: 24.8 ms]
Latency max : 273.9 ms [insert: 273.9 ms]
Total partitions : 15,000,000 [insert: 15,000,000]
Total errors : 0 [insert: 0]
Total GC count : 33
Total GC memory : 343.809 GiB
Total GC time : 6.7 seconds
Avg GC time : 202.8 ms
StdDev GC time : 42.1 ms
Total operation time : 00:01:18 {code}
Stats about flushing (allocated - allocated in heap in the flushing thread, cpu
time - CPU time for the flushing thread, time spent - clock time)
Before:
{code}
INFO [PerDiskMemtableFlushWriter_0:18] 2025-12-18T12:20:23,452
Flushing.java:196 - Completed flushing
/u02/dmko_cassandra/data/stress/stress_table-1b255f4def2540a60000000000000005/pa-44-big-Data.db
(226.713MiB) for commitlog position CommitLogPosition(segmentId=1766060262702,
position=236431), time spent: 5180 ms, bytes flushed: 237725439/(47545087 per
sec), partitions flushed: 258681/(51736 per sec), cpu time: 4074 ms, allocated:
954.056MiB
{code}
After:
{code}
INFO [PerDiskMemtableFlushWriter_0:11] 2025-12-18T20:08:42,743
Flushing.java:196 - Completed flushing
/u02/dmko_cassandra/data/stress/stress_table-1b255f4def2540a60000000000000005/pa-37-big-Data.db
(226.686MiB) for commitlog position CommitLogPosition(segmentId=1766088439199,
position=165252), time spent: 3767 ms, bytes flushed: 237697547/(79232515 per
sec), partitions flushed: 258813/(86271 per sec), rows: 2588130/(862710 per
sec), cpu time: 2581 ms, allocated: 204.196MiB
{code}
> Optimize memtable flush logic
> -----------------------------
>
> Key: CASSANDRA-21083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21083
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Local/Memtable
> Reporter: Dmitry Konstantinov
> Assignee: Dmitry Konstantinov
> Priority: Normal
> Fix For: 5.x
>
>
> Memtable flushing to disk impacts write performance and can be a limiting
> factor for write throughput:
> * If we cannot flush fast enough we have to limit writes to memtables due to
> lack of available memory for them
> * flushing logic can be CPU-intensive and complete with writing threads for
> CPU by stealing 1-2 cores (or even more if memtable_flush_writers is set to a
> higher value)
> More details about suggested optimisations will be added later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
