On Thu, Feb 27, 2014 at 2:32 PM, Marco <ma...@mashape.com> wrote: > Hi Ector, > > RAID 1 would cut your per node disk capacity in half (1.5TB), so at 4 > nodes, your overall cluster capacity would only be 6TB. > > As I mentioned every server has two 3TB HDD (total 6TB), so the cluster size > would effectively be 12TB in a Raid 1 configuration.
You are correct. I misread your original sentence. > The biggest issue here is that with both Raid 1 and a replication factor of > 2, every piece of data is going to be stored four times the cluster. Another > solution could be a replication factor of 2 with Raid 0. If an HDD fails I > will lose the machine, but I can safely bring it up again since the data has > been already stored in the other nodes. Yes, but consider that in the last case you can't perform any maintenance on the cluster until that machine is fully restored (without making a subset of your data unavailable). I think that each form of replication brings different advantages to the table: - RAID lets you survive disk failures without crashing the file system - RAID also allows you to recover quicker in the event of a disk failure - Riak replicas give you the ability to survive machine failures and remain available during network partitions A lot depends on the type of availability you want to give up when failures occur. > On Thu, Feb 27, 2014 at 6:18 AM, Hector Castro <hec...@basho.com> wrote: >> >> Hi Marco, >> >> Please see my responses inline below. >> >> -- >> Hector >> >> >> On Thu, Feb 27, 2014 at 3:04 AM, Marco <ma...@mashape.com> wrote: >> > Hi everybody, >> > >> > I have a couple of questions regarding the performance requirements for >> > a >> > 12TB Riak cluster, specifically: >> >> Have you taken the default replication factor of 3 here, or is this >> the sum of your data? >> >> > 1) I wonder if four servers, each one with the following spec, can >> > handle >> > it: 6 cores at 2.20GHz (15MB cache), 16GB of memory, and two 3TB HDD in >> > a >> > Raid 1 configuration. And if not, what would be the recommended number >> > of >> > machines we should provision. These machines are physical machines. We >> > execute one or two M/R jobs per day, the scripts are written in Erlang. >> >> For production Riak deployments, we generally recommend a cluster with >> 5 nodes or greater. [0] >> >> Beyond that, some decisions around backend, overall capacity, and >> access patterns (anything outside of M/R) need to be taken into >> consideration before validating a set of hardware specifications. A >> high level guide of the decisions in the process are outlined here: >> >> - http://docs.basho.com/riak/latest/ops/building/planning/system-planning/ >> - http://docs.basho.com/riak/latest/ops/building/planning/cluster/ >> >> > 2) Since we have two HDD per server in a Raid 1 configuration, I also >> > wonder >> > if we can safely set the replication factor to one, since the data is >> > already stored twice: if we lose an HDD, we still got the data on the >> > other >> > one thanks to the Raid 1 conf. >> >> RAID 1 would cut your per node disk capacity in half (1.5TB), so at 4 >> nodes, your overall cluster capacity would only be 6TB. >> >> For the N=1 question, I guess the best way to answer it is that Riak >> maintains replicas within the system to facilitate high availability. >> Any node or disk outage (or even scheduled maintenance) is going to >> cause replicas to be unavailable in your cluster. This is generally >> not desirable in any clustered configuration. >> >> > Thanks >> >> [0] >> http://basho.com/why-your-riak-cluster-should-have-at-least-five-nodes/ >> >> > _______________________________________________ >> > riak-users mailing list >> > riak-users@lists.basho.com >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > > > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
