Leave the RF at 3. Especially since you use write ALL consistency. It's actually a really bad idea to have your RF set to the same value as the number of nodes you have. If one of your nodes goes down, your writes will fail. In fact I would suggest leaving your RF at 3 and set read and write consistency to quorum.
On Thu, Apr 24, 2014 at 8:21 AM, Katriel Traum <katr...@google.com> wrote: > Hello list, > > I have a cluster of 3 nodes with RF=3. The cluster load is daily bulk > write/delete/compact, and read the rest of the time. > For better read performance, and to make sure data is 100% consistent, we > write with "ALL" and read "ONE", stopping the write process if there is a > problem. > > My problem is that I've maxed out my network cards. I do not have separate > cards for inter-node communication. > The knee jerk reaction was adding more nodes, but I'm not sure what > replication factor to set, 3 or 6. > My first thought was to leave it at 3, but having 6 nodes, means that if > the co-ordinator does not have the data, it will read it from another node: > 1. Data Node -> Coordinator > 2. Coordinator -> Client > This effectively means that I double the amount of data going through the > system: > > Setting it to RF=6, means that every node will always have a replica of > the data, and reading with "ONE", will always be contained in the > co-ordinator. The downside is I loose the added value of redundancy during > the write cycle. > > Does anyone have an insight or idea if my assumptions are correct? Does > inter-node communication really add all this network overhead? > > Thanks, > Katriel > > -- John Pyeatt Singlewire Software, LLC www.singlewire.com ------------------ 608.661.1184 john.pye...@singlewire.com
