[
https://issues.apache.org/jira/browse/CASSANDRA-21019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-21019:
-------------------------------------------
Fix Version/s: 4.0.22
4.1.13
5.0.10
6.0
6.0-alpha3
7.0
(was: 4.0.x)
(was: 4.1.x)
(was: 5.0.x)
(was: 7.x)
(was: 6.0.x)
Source Control Link:
https://github.com/apache/cassandra/commit/2152f78dbafa26f8a9838b4b86825bfd1f82d514
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
approvals on https://github.com/apache/cassandra/pull/5049 (and on
https://github.com/datastax/cassandra/pull/2520 )
> Memtable allocator: separate memory usage tracking and limit checking
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-21019
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21019
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Local/Memtable
> Reporter: Dmitry Konstantinov
> Assignee: Michael Semb Wever
> Priority: High
> Fix For: 4.0.22, 4.1.13, 5.0.10, 6.0, 6.0-alpha3, 7.0
>
> Attachments: ci_summary_thelastpickle_mck-21019-5.0_175.html,
> ci_summary_thelastpickle_mck-21019-6.0_101.html,
> ci_summary_thelastpickle_mck-21019-trunk_653.html,
> results_details_thelastpickle_mck-21019-5.0_175.tar.xz,
> results_details_thelastpickle_mck-21019-6.0_101.tar.xz,
> results_details_thelastpickle_mck-21019-trunk_653.tar.xz
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> The following optimization idea has been suggested by @blambov in
> CASSANDRA-20226:
> {quote}
> There's another option to consider here. The allocation mechanism does not
> need to check the limit for individual cell writes. We could just as well
> track the usage of a mutation in a single {{allocate}} call after it
> completes, or track the allocations with a {{LongAdder}} without checking if
> the limit is hit, and check if we need to wait for room before starting to
> apply a mutation.
> We use the {{allocate}} code to decide:
> - whether to initiate a flush, when the chosen memory limit is filled to
> some ratio
> - whether to pause accepting writes, when the chosen memory limit has been
> exhausted
> For the former use there is absolutely no benefit to make these decisions at
> the individual allocation level, as we will wait for the mutation to complete
> anyway before flushing anything. For the latter, I'd argue that the
> allocation-level tracking is actually hurting us. The reason for this is that
> we can have the limit be hit at any time during the application of a
> mutation, holding multiple locks (which necessitates the complexity of the
> {{isBlocking}} mutation signal), a partial copy of the mutation already
> written to the memtable structures, and a likely expanded version of the
> mutation to be applied on heap, keeping hold of more total memory than we
> would if we allowed the operation to continue.
> If, instead, we check the allocation limits _before starting_ a mutation and,
> once started, allow it to fully progress to completion, we can avoid this
> situation at the cost of being somewhat late to notice that the limit has
> been reached. This means that the limit will be breached, but this also
> happens as it stands now because we will permit operations to run to
> completion if the memtable they have been marked for is scheduled for a flush
> – which is effectively the same thing as not having noticed the memory limit
> would be breached by this mutation at the time when we decided to start it.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
