I'm using cassandra 1.0. Been doing some testing on using cass's cache. When I turn it on (using the CLI) I see ParNew jump from 3-4ms to 200-300ms. This really screws with response times, which jump from ~25-30ms to 1300+ms. I've increase new gen and that helps, but still this is suprising to me, especially since 1.0 defaults to the SerializingCacheProvider – off heap.
The interesting tid bit is that I have wide rows. 70k+ columns per row, ~50 bytes per column value. The cache only must be about 400 rows to catch all the data per node and JMX is reporting 100% cache hits. Nodetool ring reports < 2gb per node, my heap is 6gb and total RAM is 16gb. Thoughts?
