There is no reason to change the RF on the system keyspace, it should probably not be allowed.
The system keyspace uses a LocalPartitioner and it's data is not replicated through the same mechanism as a user keyspace. Aaron On 31 Mar 2011, at 10:22, Jeremy Stribling wrote: > On 03/30/2011 02:54 PM, Jeremy Stribling wrote: >> After restarting a Cassandra 0.7.2 node, the node catches an exception >> during initialization and refuses to start: >> >> Caused by: org.apache.cassandra.config.ConfigurationException: Attempt to >> assign id to existing column family. >> at org.apache.cassandra.config.CFMetaData.map(CFMetaData.java:222) >> at >> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:477) >> >> ... 2 more >> >> Unlike a previous thread about this topic >> (http://www.mail-archive.com/user@cassandra.apache.org/msg09024.html), we >> are not trying to preserve the JVM across restarts. The restart comes up in >> an entirely fresh JVM. We are, however, embedding Cassandra in our >> application, but we're using the same steps used by AbstractCassandraDaemon >> to bring it up. >> >> Looking briefly through the code, the only way I see that this can happen is >> if loadSchemas tries to load information about the system table from storage >> (because the system table can be created in CFMetaData from the earlier >> DatabaseDescriptor.getTableMetaData(Table.SYSTEM_TABLE).values() call). Or >> I guess the data on disk could have multiple entries under the same key, but >> the system table issue seems more likely to me. Unfortunately the logging >> is not specific enough for me to tell which key it is failing with, and I >> haven't been able to reproduce this yet. >> >> One relevant piece of information might be that, before the restart, our >> application changed the replication factor of all the tables, including the >> system table: >> >> 2011-03-29 23:09:39,194 291146 [MigrationStage:1] INFO >> org.apache.cassandra.db.migration.Migration - Applying migration >> 9f371026-5a59-11e0-b23f-65ed1eced995 Update keyspace systemrep factor:1rep >> strategy:LocalStrategy{...} to systemrep factor:3rep >> strategy:LocalStrategy{...} >> >> We're doing this in order to dynamically change the replication factor as >> new nodes are being added to the cluster (e.g., it starts off with one node >> and a repfactor of 1, and once there are three nodes, it increases the >> repfactor on all tables to 3). Is it possible that migrations over the >> system table get written to disk in a way that would cause loadSchemas() >> during a restart to hit this exception? Are we even allowed to change the >> replication factor of the system table? >> > > I've confirmed that this happens when loading column family "IndexInfo" from > the table "system" during the loadSchemas() call. Does anyone know if > there's a way to get around this? Perhaps, like I theorized, it's not legit > to change the replication factor on the system table. > >
