Is there anything going on in the logs ? Are nodes going up and down ? Can you see any messages about delivering hints ?
If the query to read the hints errors it will log "HintsCF getEPPendingHints timed out" at INFO level. Also checking, do the hinted_handoff_* settings in cassandra.yaml have their default settings ? Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 15/03/2012, at 8:35 AM, Bryce Godfrey wrote: > Forgot to mention that this is on 1.0.8 > > From: Bryce Godfrey [mailto:bryce.godf...@azaleos.com] > Sent: Wednesday, March 14, 2012 12:34 PM > To: user@cassandra.apache.org > Subject: Large hints column family > > The system HintsColumnFamily seems large in my cluster, and I want to track > down why that is. I try invoking “listEndpointsPendingHints()” for > o.a.c.db.HintedHandoffManager and it never returns, and also freezes the node > that its invoked against. It’s a 3 node cluster, and all nodes have been up > and running without issue for a while. Any help on where to start with this? > > Column Family: HintsColumnFamily > SSTable count: 11 > Space used (live): 11271669539 > Space used (total): 11271669539 > Number of Keys (estimate): 1408 > Memtable Columns Count: 338 > Memtable Data Size: 0 > Memtable Switch Count: 1 > Read Count: 3 > Read Latency: 4354.669 ms. > Write Count: 848 > Write Latency: 0.029 ms. > Pending Tasks: 0 > Bloom Filter False Postives: 0 > Bloom Filter False Ratio: 0.00000 > Bloom Filter Space Used: 12656 > Key cache capacity: 14 > Key cache size: 11 > Key cache hit rate: 0.6666666666666666 > Row cache: disabled > Compacted row minimum size: 105779 > Compacted row maximum size: 7152383774 > Compacted row mean size: 590818614 > > Thanks, > Bryce
