trying running nodetool repair on 10.80.161.101 and then cleanup on 10.102.37.168 if everything is ok.
Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 16/03/2012, at 6:45 AM, Thorsten von Eicken wrote: > I added a second node to a single-node ring. RF=1. I can't get the new > node to receive any data. Logs look fine. Here's what nodetool reports: > > # nodetool -h localhost ring > Address DC Rack Status State Load > Owns Token > > 85070591730234615865843651857942052864 > 10.102.37.168 datacenter1 rack1 Up Normal 807.81 GB > 50.00% 0 > 10.80.161.101 datacenter1 rack1 Up Normal 1.15 MB > 50.00% 85070591730234615865843651857942052864 > > Just a "little" imbalance. Yes, I use partitioner: > org.apache.cassandra.dht.RandomPartitioner > I tried moving the new node's token up/down by 1 and it triggers the log > messages you'd expect, but no data gets transferred. How do I > troubleshoot this? Below are the log messages I see when restarting the > new node: > > INFO [main] 2012-03-15 17:31:08,616 AbstractCassandraDaemon.java (line > 120) JVM vendor/version: > Java HotSpot(TM) 64-Bit Server VM/1.6.0_24 > INFO [main] 2012-03-15 17:31:14,812 CommitLog.java (line 178) Log > replay complete, 8 replayed mutations > INFO [main] 2012-03-15 17:31:14,825 StorageService.java (line 390) > Cassandra version: 1.0.6 > INFO [main] 2012-03-15 17:31:14,825 StorageService.java (line 391) > Thrift API version: 19.19.0 > INFO [main] 2012-03-15 17:31:14,825 StorageService.java (line 404) > Loading persisted ring state > INFO [main] 2012-03-15 17:31:14,834 StorageService.java (line 482) > Starting up server gossip > INFO [main] 2012-03-15 17:31:15,372 MessagingService.java (line 247) > Starting Encrypted Messaging Service on SSL port 7000 > INFO [main] 2012-03-15 17:31:15,376 MessagingService.java (line 268) > Starting Messaging Service on port 7001 > INFO [main] 2012-03-15 17:31:15,401 StorageService.java (line 579) > Using saved token 85070591730234615865843651857942052864 > INFO [main] 2012-03-15 17:31:15,402 ColumnFamilyStore.java (line 692) > Enqueuing flush of Memtable-LocationInfo@645492252(53/66 serialized/live > bytes, 2 ops) > INFO [FlushWriter:1] 2012-03-15 17:31:15,403 Memtable.java (line 240) > Writing Memtable-LocationInfo@645492252(53/66 serialized/live bytes, 2 ops) > INFO [FlushWriter:1] 2012-03-15 17:31:15,421 Memtable.java (line 277) > Completed flushing /mnt/ebs/data/system/LocationInfo-hc-32-Data.db (163 > bytes) > INFO [main] 2012-03-15 17:31:15,424 StorageService.java (line 948) Node > /10.80.161.101 state jump to normal > INFO [main] 2012-03-15 17:31:15,434 StorageService.java (line 589) > Bootstrap/Replace/Move completed! Now serving reads. > > # describe keyspace > Keyspace: rslog_production: > Replication Strategy: org.apache.cassandra.locator.SimpleStrategy > Durable Writes: true > Options: [replication_factor:1] > Column Families: >
