[
https://issues.apache.org/jira/browse/CASSANDRA-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick McFadin reassigned CASSANDRA-21659:
-------------------------------------------
Assignee: Patrick McFadin
> Adding a large amount of schema allocates excessively
> -----------------------------------------------------
>
> Key: CASSANDRA-21659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21659
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
> Reporter: Patrick McFadin
> Assignee: Patrick McFadin
> Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Every schema change allocates in proportion to the whole schema rather than
> to what changed.
> h2. What
> Keyspaces.diff and Tables.diff build created/dropped with filter(), which
> removes
> every non-matching entry from a copy one at a time. On a CREATE TABLE no
> keyspace
> is created or dropped, so Keyspaces.filter falls into withoutKsTablesViews and
> removes every table in the cluster individually – twice per diff. Tables.diff
> then calls TableMetadata.compare for every surviving table, and compare has no
> identity fast path.
> Several diffs run per DDL: AlterSchemaStatement:194 and :200, AlterSchema:160,
> DistributedSchema:213, :260 and :325.
> h2. Change
> * Collect created/dropped directly instead of filtering whole collections.
> * Skip comparing entries carried over by reference. Untouched TableMetadata
> are
> reference-identical across a schema change (Tables.Builder.add stores the
> instance verbatim), and x.compare(x) is empty by construction, so identity is
> exact here, not an approximation.
> No signature, format or API changes.
> h2. Result
> New KeyspacesDiffScalingTest measures allocation for adding one table at 400
> and
> 3200 existing tables and asserts the growth ratio. Allocation rather than
> elapsed
> time: counted exactly rather than sampled, independent of GC timing, and it is
> what produces the failure (a GC wall, not gradual slowdown).
> || ||400 tables||3200 tables||growth||
> |before|292,552 B|3,774,864 B|12.9x|
> |after|3,944 B|3,944 B|1.0x|
> Before is superlinear because the BTreeMap teardown is O(N log N).
> h2. Scope – read before benchmarking
> This removes the allocation term, not the scan. Time stays O(N) per diff and
> bulk
> creation stays quadratic in wall-clock. What it removes is the allocation that
> drives the GC wall. Wall-clock at small N will barely move.
> h2. Tests
> New KeyspacesDiffScalingTest: 2 scaling assertions, 3 correctness guards.
> Verified
> red before and green after, and re-verified the final test fails with the fix
> stashed. Regression: schema 27 suites/130 tests, tcm 13/55. checkstyle and
> checkstyle-test clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
