Hello Experts, I see a queer behavior from on of the Cassandra nodes in my cluster where the data is not flushed off Commitlogs and the Commitlog file grows in number. I was inserting the data into the cluster and since yesterday this node had more than 900 commitlog files.
-rw-r--r-- 1 dev dev 134218072 2010-10-05 15:11 CommitLog-1286218976629.log -rw-r--r-- 1 dev dev 134217909 2010-10-05 15:11 CommitLog-1286316625590.log -rw-r--r-- 1 dev dev 134217872 2010-10-05 15:15 CommitLog-1286316673050.log -rw-r--r-- 1 dev dev 134217957 2010-10-05 15:19 CommitLog-1286316917618.log -rw-r--r-- 1 dev dev 134217764 2010-10-05 15:21 CommitLog-1286317178922.log -rw-r--r-- 1 dev dev 134217871 2010-10-05 15:24 CommitLog-1286317311372.log -rw-r--r-- 1 dev dev 134218029 2010-10-05 15:25 CommitLog-1286317446787.log -rw-r--r-- 1 dev dev 134218065 2010-10-05 15:26 CommitLog-1286317506892.log -rw-r--r-- 1 dev dev 134218100 2010-10-05 15:27 CommitLog-1286317567927.log -rw-r--r-- 1 dev dev 134218020 2010-10-05 15:29 CommitLog-1286317634668.log -rw-r--r-- 1 dev dev 134217922 2010-10-05 15:31 CommitLog-1286317741576.log -rw-r--r-- 1 dev dev 134217944 2010-10-05 15:32 CommitLog-1286317887655.log -rw-r--r-- 1 dev dev 134217961 2010-10-05 15:33 CommitLog-1286317968395.log -rw-r--r-- 1 dev dev 134217926 2010-10-05 15:34 CommitLog-1286318033241.log -rw-r--r-- 1 dev dev 134218045 2010-10-05 15:36 CommitLog-1286318097727.log -rw-r--r-- 1 dev dev 134218027 2010-10-05 15:37 CommitLog-1286318190654.log -rw-r--r-- 1 dev dev 134218060 2010-10-05 15:38 CommitLog-1286318277987.log -rw-r--r-- 1 dev dev 134218055 2010-10-05 15:41 CommitLog-1286318338561.log -rw-r--r-- 1 dev dev 134218069 2010-10-05 15:41 CommitLog-1286318461180.log -rw-r--r-- 1 dev dev 134217791 2010-10-05 15:43 CommitLog-1286318519193.log -rw-r--r-- 1 dev dev 134218066 2010-10-05 15:44 CommitLog-1286318609960.log -rw-r--r-- 1 dev dev 134218093 2010-10-05 15:45 CommitLog-1286318692656.log It seems that commitlog files spawned every minute and when it grew over 900 files the Cassandra node went down. Now I'm trying to start my node but it has started to read all of those Commitlog files and takes huge amount of time to restart. Can anyone throw some light on this unusual behavior. Till yesterday before starting the data insertion the node had almost 550GB of data. Incidentally I did stop another node prior to inserting the data because it was 100% full. I'm using RandomPartitioner with Initial Tokens. Thanking all in advance. Rana
