[
https://issues.apache.org/jira/browse/CASSANDRA-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick McFadin updated CASSANDRA-21660:
----------------------------------------
Description:
h2. What
Tables keep three ImmutableMaps and rebuild all of them on every mutation:
with() is builder().add(this).add(table).build(), withSwapped() is
without().with(). A single CREATE TABLE therefore copies every existing table
into three fresh maps more than once, and Builder.add recomputes index metadata
for every indexed table while doing so.
CEP-21 converted the outer collection, Keyspaces, to a persistent BTreeMap. TCM
holds schema at several epochs at once, so copy-on-write would copy the world
per epoch. The collections nested inside a keyspace were not converted.
h2. Change
* Back the three maps with BTreeMap, so with/without/withSwapped update rather
than copy.
* Maintain index tables incrementally, keyed off IndexMetadata.name (provably
the same key: indexTableName is base + '.' + index.name).
* Builder keeps the duplicate-name rejection ImmutableMap.Builder.build() gave.
h2. Result
||operation||400 before||3200 before||growth||400 after||3200 after||growth||
|with|218,951 B|1,687,617 B|7.7x|5,728 B|5,920 B|1.0x|
|withSwapped|425,824 B|3,349,656 B|7.9x|1,936 B|2,448 B|1.3x|
|without|213,896 B|1,682,368 B|7.9x|928 B|1,200 B|1.3x|
The residual 1.3x is the O(log n) a persistent map allocates per update.
h2. REVIEWERS: requires a clean build
Tables.indexTables() narrows from ImmutableMap to Map. That is binary
incompatible, and ant's timestamp-based javac will not recompile callers whose
source did not change. With a stale build/ you get a NoSuchMethodError inside
SecondaryIndexManager.createIndex, which LocalLog.processPendingInternal
catches and logs as "Could not process the entry" – leaving an index silently
unregistered and surfacing later as an unrelated NPE. Run ant clean first. No
source change can prevent this.
h2. Tests
New TablesScalingTest: 3 allocation assertions, 4 correctness guards
(add/remove/ replace semantics; iteration deterministic and agreeing with
lookup by name and id). Regression on a clean build: schema 29 suites, tcm 13
suites, index 3 suites /61 tests, 0 failures. Checkstyle and checkstyle-test
clean.
was:
h2. What
Tables keep three ImmutableMaps and rebuild all of them on every mutation:
with() is builder().add(this).add(table).build(), withSwapped() is
without().with(). A single CREATE TABLE therefore copies every existing table
into three fresh maps more than once, and Builder.add recomputes index metadata
for every indexed table while doing so.
CEP-21 converted the outer collection, Keyspaces, to a persistent BTreeMap.
TCM holds schema at several epochs at once, so copy-on-write would copy the
world
per epoch. The collections nested inside a keyspace were not converted.
h2. Change
* Back the three maps with BTreeMap, so with/without/withSwapped update rather
than copy.
* Maintain index tables incrementally, keyed off IndexMetadata.name (provably
the
same key: indexTableName is base + '.' + index.name).
* Builder keeps the duplicate-name rejection ImmutableMap.Builder.build() gave.
h2. Result
||operation||400 before||3200 before||growth||400 after||3200 after||growth||
|with|218,951 B|1,687,617 B|7.7x|5,728 B|5,920 B|1.0x|
|withSwapped|425,824 B|3,349,656 B|7.9x|1,936 B|2,448 B|1.3x|
|without|213,896 B|1,682,368 B|7.9x|928 B|1,200 B|1.3x|
The residual 1.3x is the O(log n) a persistent map allocates per update.
h2. REVIEWERS: requires a clean build
Tables.indexTables() narrows from ImmutableMap to Map. That is binary
incompatible, and ant's timestamp-based javac will not recompile callers whose
source did not change. With a stale build/ you get a NoSuchMethodError inside
SecondaryIndexManager.createIndex, which LocalLog.processPendingInternal catches
and logs as "Could not process the entry" – leaving an index silently
unregistered and surfacing later as an unrelated NPE. Run ant clean first. No
source change can prevent this.
h2. Tests
New TablesScalingTest: 3 allocation assertions, 4 correctness guards
(add/remove/
replace semantics; iteration deterministic and agreeing with lookup by name and
id). Regression on a clean build: schema 29 suites, tcm 13 suites, index 3
suites
/61 tests, 0 failures. Checkstyle and checkstyle-test clean.
> Back Tables with a persistent map so schema changes do not copy the collection
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-21660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21660
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
> Reporter: Patrick McFadin
> Assignee: Patrick McFadin
> Priority: Normal
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. What
> Tables keep three ImmutableMaps and rebuild all of them on every mutation:
> with() is builder().add(this).add(table).build(), withSwapped() is
> without().with(). A single CREATE TABLE therefore copies every existing table
> into three fresh maps more than once, and Builder.add recomputes index
> metadata for every indexed table while doing so.
> CEP-21 converted the outer collection, Keyspaces, to a persistent BTreeMap.
> TCM holds schema at several epochs at once, so copy-on-write would copy the
> world per epoch. The collections nested inside a keyspace were not converted.
> h2. Change
> * Back the three maps with BTreeMap, so with/without/withSwapped update
> rather than copy.
> * Maintain index tables incrementally, keyed off IndexMetadata.name
> (provably the same key: indexTableName is base + '.' + index.name).
> * Builder keeps the duplicate-name rejection ImmutableMap.Builder.build()
> gave.
> h2. Result
> ||operation||400 before||3200 before||growth||400 after||3200 after||growth||
> |with|218,951 B|1,687,617 B|7.7x|5,728 B|5,920 B|1.0x|
> |withSwapped|425,824 B|3,349,656 B|7.9x|1,936 B|2,448 B|1.3x|
> |without|213,896 B|1,682,368 B|7.9x|928 B|1,200 B|1.3x|
> The residual 1.3x is the O(log n) a persistent map allocates per update.
> h2. REVIEWERS: requires a clean build
> Tables.indexTables() narrows from ImmutableMap to Map. That is binary
> incompatible, and ant's timestamp-based javac will not recompile callers
> whose source did not change. With a stale build/ you get a NoSuchMethodError
> inside SecondaryIndexManager.createIndex, which
> LocalLog.processPendingInternal catches and logs as "Could not process the
> entry" – leaving an index silently unregistered and surfacing later as an
> unrelated NPE. Run ant clean first. No source change can prevent this.
> h2. Tests
> New TablesScalingTest: 3 allocation assertions, 4 correctness guards
> (add/remove/ replace semantics; iteration deterministic and agreeing with
> lookup by name and id). Regression on a clean build: schema 29 suites, tcm 13
> suites, index 3 suites /61 tests, 0 failures. Checkstyle and checkstyle-test
> clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
