[
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
graham sanderson updated CASSANDRA-7546:
----------------------------------------
Attachment: 7546.20_alt.txt
Thanks for taking a look Benedict - I am in the process of actually measuring
the effect via cassandra stress in the real world for code paths that might hit
this (actual user initiated contended write on a partition, hint based
contented write, and low cardinality secondary indexes).
I had initially shied away from the non obviously paired monitor usage because
a) it requires unsafe, and b) whilst monitorenter and monitorexit do exist as
separate byte codes, I seem to recall that there is code in the monitor fast
path (especially w.r.t. biased locking which we probably wan't to avoid) that
expects them to be trivially paired... c) on the negative side this forces two
loops under contention, but that is better than unbounded obviously.
I'm not sure if b) is an actual issue (or what effect it has - does it matter
if the monitor is always inflated for example).
As for c) I guess I'll have to measure and see.
Note I attached the current version of the 2.0 patch I was testing with, which
pushed the loop body into a class similar to ColumnUpdater in 2.1 (which I
would update accordingly)...
again any attempt to further simplify the three separate loops, didn't really
make things any simpler looking.
I'm not sure my code (whilst not blazingly pretty) is insanely hard to reason
about... clearly someone always makes progress (since any change in state is
dependent on a CAS success or failure due to someone else succeeding); the
un-contended and highly contended cases are pretty simple, and any
flip-flopping between the "count" and "raw" state again always means we made
progress AND must (still) be under low contention anyway.
In any case, I'll defer to your judgement as to what you'd like to see in the
code base... as I say some actual numbers outside of a synthetic test should
help
> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -----------------------------------------------------------------------------
>
> Key: CASSANDRA-7546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: graham sanderson
> Assignee: graham sanderson
> Attachments: 7546.20.txt, 7546.20_alt.txt, suggestion1.txt,
> suggestion1_21.txt
>
>
> In order to preserve atomicity, this code attempts to read, clone/update,
> then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some
> fairly staggering memory growth (the more cores on your machine the worst it
> gets).
> Whilst many usage patterns don't do highly concurrent updates to the same
> partition, hinting today, does, and in this case wild (order(s) of magnitude
> more than expected) memory allocation rates can be seen (especially when the
> updates being hinted are small updates to different partitions which can
> happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation
> whilst not slowing down the very common un-contended case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
