sepuri sai krishna created CASSANDRA-21549:
----------------------------------------------
Summary: Masked columns cannot be deserialized from cluster
metadata, preventing node startup
Key: CASSANDRA-21549
URL: https://issues.apache.org/jira/browse/CASSANDRA-21549
Project: Apache Cassandra
Issue Type: Bug
Components: Feature/Dynamic Data Masking, Transactional Cluster
Metadata
Reporter: sepuri sai krishna
Assignee: sepuri sai krishna
h2. Summary
{{ColumnMask.Serializer.deserialize}} has two independent defects that make any
table with a
dynamic data masked column unreadable from a serialized {{ClusterMetadata}}. A
node that has taken
a cluster metadata snapshot then fails to start.
h2. Details
In {{src/java/org/apache/cassandra/cql3/functions/masking/ColumnMask.java}}:
{code:java}
List<AbstractType<?>> argTypes = new ArrayList<>(numArgs + 1);
argTypes.set(0, columnType); // (1)
ByteBuffer[] partialArgValues = new ByteBuffer[numArgs];
for (int i = 0; i < numArgs; i++)
{
AbstractType<?> argType = CQLTypeParser.parse(keyspace, in.readUTF(),
types);
argTypes.set(i + 1, argType); // (1)
boolean valuePresent = in.readBoolean();
partialArgValues[i] = valuePresent ? null :
ByteBufferUtil.readWithVIntLength(in); // (2)
}
{code}
# {{new ArrayList<>(numArgs + 1)}} sets the *capacity*, not the size, so the
list is empty and {{set(0, ..)}}
throws {{IndexOutOfBoundsException}}. This fires for every masked column,
including
{{MASKED WITH DEFAULT}}.
# The ternary is inverted: the argument is discarded when present, and a read
is attempted when
absent. Besides losing the value this desynchronises the input stream,
corrupting every field
deserialized afterwards. With defect (1) fixed in isolation, the same tests
fail with
{{EOFException: EOF after 43 bytes out of 1024}}.
h2. Impact
{{ColumnMask.serializer}} is reached through
{{ColumnMetadata -> TableMetadata -> Tables -> KeyspaceMetadata ->
DistributedSchema -> ClusterMetadata}}.
Ordinary DDL does not hit it, because schema changes propagate as
transformations that each node
replays locally. It is reached when the schema is embedded in a serialized
{{ClusterMetadata}},
i.e. for cluster metadata snapshots, which are read back on startup
({{LocalLog.replayPersisted}}) and when a lagging peer catches up from the CMS.
Restarting a node that has a masked column and a stored snapshot fails during
startup:
{code}
java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at java.base/java.util.ArrayList.set(ArrayList.java:470)
at
org.apache.cassandra.cql3.functions.masking.ColumnMask$Serializer.deserialize(ColumnMask.java:318)
at
org.apache.cassandra.schema.ColumnMetadata$Serializer.deserialize(ColumnMetadata.java:819)
at
org.apache.cassandra.schema.TableMetadata$Serializer.deserialize(TableMetadata.java:2184)
at
org.apache.cassandra.schema.Tables$Serializer.deserialize(Tables.java:317)
at
org.apache.cassandra.schema.KeyspaceMetadata$Serializer.deserialize(KeyspaceMetadata.java:526)
at
org.apache.cassandra.schema.DistributedSchema$Serializer.deserialize(DistributedSchema.java:487)
at
org.apache.cassandra.tcm.ClusterMetadata$Serializer.deserialize(ClusterMetadata.java:1326)
at
org.apache.cassandra.tcm.MetadataSnapshots.fromBytes(MetadataSnapshots.java:64)
at
org.apache.cassandra.tcm.MetadataSnapshots$SystemKeyspaceMetadataSnapshots.getLatestSnapshot(MetadataSnapshots.java:131)
at
org.apache.cassandra.tcm.log.SystemKeyspaceStorage.getPersistedLogState(SystemKeyspaceStorage.java:133)
at
org.apache.cassandra.tcm.log.LocalLog.replayPersisted(LocalLog.java:584)
at
org.apache.cassandra.tcm.Startup.initializeAsNonCmsNode(Startup.java:185)
at org.apache.cassandra.tcm.Startup.initialize(Startup.java:120)
{code}
Note that {{MetadataSnapshots.getSnapshot}} only catches {{IOException}}, so
this unchecked
exception propagates rather than degrading to a null snapshot.
h2. Reproduction
{code}
CREATE TABLE ks.t (k int PRIMARY KEY, v text MASKED WITH mask_inner(2, 1));
nodetool cms snapshot
# restart the node -> startup fails with the stack above
{code}
h2. Affected versions
Present on {{cassandra-6.0}} and {{trunk}}. Not present on {{cassandra-5.0}},
which has no
{{ColumnMask.Serializer}}; dynamic data masking shipped in 5.0 but the schema
is only serialized
into cluster metadata from 6.0 onwards. Patch is therefore based on
{{cassandra-6.0}}.
h2. Tests
Existing DDM tests do not cover this because none of them force a metadata
snapshot. Added:
* three round-trip cases in {{SchemaMetadataSerializationTest}} covering a
masking function with no
partial arguments, with partial arguments, and with a null partial argument
* a new dtest {{ColumnMaskMetadataSnapshotTest}} covering snapshot read-back
and node restart
All fail before the patch and pass after it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
