We're on Cassandra 3.7, running on Ubuntu 14.04. In terms of system utilization, we saw one Cassandra process which was using 100% CPU, but overall load was very low on the box. Disk utilization was largely nominal.
On Wed, Nov 16, 2016 at 2:19 PM, Jeff Jirsa <jeff.ji...@crowdstrike.com> wrote: > What version? > > Is the system doing anything (do you see high CPU / disk usage)? > > > > Sometimes restarts will trigger some changes to files on disk that are > mostly invisible in the logs (https://issues.apache.org/ > jira/browse/CASSANDRA-11163 for example), but it’s usually during a > different part of the startup process (you’d be seeing different log > messages), and would eventually complete. > > > > > > > > *From: *Daniel Subak <daniel.su...@klaviyo.com> > *Reply-To: *"user@cassandra.apache.org" <user@cassandra.apache.org> > *Date: *Wednesday, November 16, 2016 at 11:05 AM > *To: *"user@cassandra.apache.org" <user@cassandra.apache.org> > *Subject: *Cassandra Node Restart Stuck in STARTING? > > > > Hey everyone, > > Ran into an issue running a node restart where "nodetool netstats" > reported the node as "STARTING" with no streams when run locally. "nodetool > status" run on other nodes reported that node as "DN". Both of those were > expected. However, tailing the logs, there didn't seem to be anything > noteworthy happening (below are the last few log lines in system.log.) Has > anyone seen this behavior before? We'd love to be able to better monitor > what is happening during a restart if anyone has some information on what > happens during this phase! Happy to provide more info if needed, but even a > high level general explanation would provide some clarity > > Thanks, > > Dan > > > INFO [main] 2016-11-16 18:07:55,907 ColumnFamilyStore.java:405 - > Initializing system_schema.keyspaces > INFO [main] 2016-11-16 18:07:55,942 ColumnFamilyStore.java:405 - > Initializing system_schema.tables > INFO [main] 2016-11-16 18:07:55,971 ColumnFamilyStore.java:405 - > Initializing system_schema.columns > INFO [main] 2016-11-16 18:07:55,992 ColumnFamilyStore.java:405 - > Initializing system_schema.triggers > INFO [main] 2016-11-16 18:07:56,010 ColumnFamilyStore.java:405 - > Initializing system_schema.dropped_columns > INFO [main] 2016-11-16 18:07:56,026 ColumnFamilyStore.java:405 - > Initializing system_schema.views > INFO [main] 2016-11-16 18:07:56,047 ColumnFamilyStore.java:405 - > Initializing system_schema.types > INFO [main] 2016-11-16 18:07:56,066 ColumnFamilyStore.java:405 - > Initializing system_schema.functions > INFO [main] 2016-11-16 18:07:56,081 ColumnFamilyStore.java:405 - > Initializing system_schema.aggregates > INFO [main] 2016-11-16 18:07:56,093 ColumnFamilyStore.java:405 - > Initializing system_schema.indexes > INFO [main] 2016-11-16 18:07:56,102 ViewManager.java:139 - Not submitting > build tasks for views in keyspace system_schema as storage service is not > initialized >
