kinds of like https://issues.apache.org/jira/browse/CASSANDRA-3733 but maybe different.
Have you recently dropped as CF ? it looks like the hints CF is only compacted if hints are replayed. If they are dropped because the CF no longer exists compaction is not forced ( https://github.com/apache/cassandra/blob/cassandra-1.1/src/java/org/apache/cassandra/db/HintedHandOffManager.java#L370 ) Can you create a ticket on https://issues.apache.org/jira/browse/CASSANDRA with the details and update the thread. Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 21/05/2012, at 8:36 PM, Arend-Jan Wijtzes wrote: > Hi, > > We are running a small test cluster and recently installed Cassandra 1.1 and > started with a new clean database. We keep seeing these messages in the log > on just one of our nodes: > > INFO [HintedHandoff:1] 2012-05-21 09:49:56,757 HintedHandOffManager.java > (line 294) Started hinted handoff for token: > 85070591730234615865843651857942052864 with IP: /10.0.0.73 > INFO [HintedHandoff:1] 2012-05-21 09:49:56,775 HintedHandOffManager.java > (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.73 > INFO [HintedHandoff:1] 2012-05-21 09:59:56,756 HintedHandOffManager.java > (line 294) Started hinted handoff for token: > 42535295865117307932921825928971026432 with IP: /10.0.0.69 > INFO [HintedHandoff:1] 2012-05-21 09:59:56,757 HintedHandOffManager.java > (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.69 > INFO [HintedHandoff:1] 2012-05-21 09:59:56,757 HintedHandOffManager.java > (line 294) Started hinted handoff for token: > 85070591730234615865843651857942052864 with IP: /10.0.0.73 > INFO [HintedHandoff:1] 2012-05-21 09:59:56,775 HintedHandOffManager.java > (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.73 > INFO [HintedHandoff:1] 2012-05-21 10:09:56,757 HintedHandOffManager.java > (line 294) Started hinted handoff for token: > 42535295865117307932921825928971026432 with IP: /10.0.0.69 > INFO [HintedHandoff:1] 2012-05-21 10:09:56,758 HintedHandOffManager.java > (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.69 > INFO [HintedHandoff:1] 2012-05-21 10:09:56,758 HintedHandOffManager.java > (line 294) Started hinted handoff for token: > 85070591730234615865843651857942052864 with IP: /10.0.0.73 > INFO [HintedHandoff:1] 2012-05-21 10:09:56,879 HintedHandOffManager.java > (line 382) Finished hinted handoff of 0 rows to endpoint /10.0.0.73 > > > All four nodes are up: > > -bash-4.1$ nodetool ring -h localhost > Note: Ownership information does not include topology, please specify a > keyspace. > Address DC Rack Status State Load Owns > Token > > 127605887595351923798765477786913079296 > 10.0.0.65 datacenter1 rack1 Up Normal 244.41 MB 25.00% > 0 > 10.0.0.69 datacenter1 rack1 Up Normal 155.39 MB 25.00% > 42535295865117307932921825928971026432 > 10.0.0.73 datacenter1 rack1 Up Normal 220.42 MB 25.00% > 85070591730234615865843651857942052864 > 10.0.0.77 datacenter1 rack1 Up Normal 296.14 MB 25.00% > 127605887595351923798765477786913079296 > > > This has been going on for days. Note that it's just two key's in the log > that keep repeating. No recent messages about HintedHandOff in the logs > on the other nodes. > > Let me know if you need more info. > > Arend-Jan > > -- > Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl
