I am planning on setting up a Cassandra cluster on a small 16 node cluster (possibly 32 way). Each machine has 8 cores 32 Gig of ram and 8 hds. My first thought is to setup one of those hds for the commit log, 6 for data and leave one for the OS. However I do have a concern about best utilizing my memory, should I run a larger heap? Should I run several cassandra processes on the same box?
My concern about the larger heap is because GC's typically get slower. And if I run several procs, does cassandra realize that it's the same box for replication purposes? I do have other hd conf options, hardware RAID 0,1,or 5. Just looking for some general configuration options as well as some real world successes with similarly sized hardware. Thanks! Aaron