Hey everyone, Ran into an issue running a node restart where "nodetool netstats" reported the node as "STARTING" with no streams when run locally. "nodetool status" run on other nodes reported that node as "DN". Both of those were expected. However, tailing the logs, there didn't seem to be anything noteworthy happening (below are the last few log lines in system.log.) Has anyone seen this behavior before? We'd love to be able to better monitor what is happening during a restart if anyone has some information on what happens during this phase! Happy to provide more info if needed, but even a high level general explanation would provide some clarity
Thanks, Dan INFO [main] 2016-11-16 18:07:55,907 ColumnFamilyStore.java:405 - Initializing system_schema.keyspaces INFO [main] 2016-11-16 18:07:55,942 ColumnFamilyStore.java:405 - Initializing system_schema.tables INFO [main] 2016-11-16 18:07:55,971 ColumnFamilyStore.java:405 - Initializing system_schema.columns INFO [main] 2016-11-16 18:07:55,992 ColumnFamilyStore.java:405 - Initializing system_schema.triggers INFO [main] 2016-11-16 18:07:56,010 ColumnFamilyStore.java:405 - Initializing system_schema.dropped_columns INFO [main] 2016-11-16 18:07:56,026 ColumnFamilyStore.java:405 - Initializing system_schema.views INFO [main] 2016-11-16 18:07:56,047 ColumnFamilyStore.java:405 - Initializing system_schema.types INFO [main] 2016-11-16 18:07:56,066 ColumnFamilyStore.java:405 - Initializing system_schema.functions INFO [main] 2016-11-16 18:07:56,081 ColumnFamilyStore.java:405 - Initializing system_schema.aggregates INFO [main] 2016-11-16 18:07:56,093 ColumnFamilyStore.java:405 - Initializing system_schema.indexes INFO [main] 2016-11-16 18:07:56,102 ViewManager.java:139 - Not submitting build tasks for views in keyspace system_schema as storage service is not initialized