Patrick McFadin created CASSANDRA-21659:
-------------------------------------------
Summary: Adding a large amount of schema allocates excessively
Key: CASSANDRA-21659
URL: https://issues.apache.org/jira/browse/CASSANDRA-21659
Project: Apache Cassandra
Issue Type: Improvement
Components: Cluster/Schema
Reporter: Patrick McFadin
Every schema change allocates in proportion to the whole schema rather than to
what changed.
h2. What
Keyspaces.diff and Tables.diff build created/dropped with filter(), which
removes
every non-matching entry from a copy one at a time. On a CREATE TABLE no
keyspace
is created or dropped, so Keyspaces.filter falls into withoutKsTablesViews and
removes every table in the cluster individually – twice per diff. Tables.diff
then calls TableMetadata.compare for every surviving table, and compare has no
identity fast path.
Several diffs run per DDL: AlterSchemaStatement:194 and :200, AlterSchema:160,
DistributedSchema:213, :260 and :325.
h2. Change
* Collect created/dropped directly instead of filtering whole collections.
* Skip comparing entries carried over by reference. Untouched TableMetadata are
reference-identical across a schema change (Tables.Builder.add stores the
instance verbatim), and x.compare(x) is empty by construction, so identity is
exact here, not an approximation.
No signature, format or API changes.
h2. Result
New KeyspacesDiffScalingTest measures allocation for adding one table at 400 and
3200 existing tables and asserts the growth ratio. Allocation rather than
elapsed
time: counted exactly rather than sampled, independent of GC timing, and it is
what produces the failure (a GC wall, not gradual slowdown).
|| ||400 tables||3200 tables||growth||
|before|292,552 B|3,774,864 B|12.9x|
|after|3,944 B|3,944 B|1.0x|
Before is superlinear because the BTreeMap teardown is O(N log N).
h2. Scope – read before benchmarking
This removes the allocation term, not the scan. Time stays O(N) per diff and
bulk
creation stays quadratic in wall-clock. What it removes is the allocation that
drives the GC wall. Wall-clock at small N will barely move.
h2. Tests
New KeyspacesDiffScalingTest: 2 scaling assertions, 3 correctness guards.
Verified
red before and green after, and re-verified the final test fails with the fix
stashed. Regression: schema 27 suites/130 tests, tcm 13/55. checkstyle and
checkstyle-test clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
