Hi Dave, On Thu, Aug 16, 2012 at 9:41 AM, Dave Brady <dbr...@weborama.com> wrote: > Thanks, Mark! > > I have a somewhat-related follow up question: is there a recommended maximum > for the amount of data held on a machine? > > I ask because at my previous company Cassandra was used, and we were advised > to put no more 250 GB per physical box. The reasoning was that in a failure > situation, rebuilding any more than this amount of data would cause too great > a performance degradation. >
So there's no hard and fast amount of data per node that you shouldn't be passing when using Riak. Rather, it makes more sense to keep in mind the resources needed when a node (or set of nodes) becomes unreachable, dead, or is sold on ebay by a sysadmin trying to raise money for a ping pong table.. Plan appropriately for capacity and Riak should operate as advertised . 250GB/node shouldn't be an issue as long as you understand the resources needed to keep the cluster going in failure and growth scenarios. (Anecdotally, I can tell you I know of one Riak user that has >5TB/physical node on a cluster with 10s of nodes.) Ryan Zezeski also pointed out that Eric Brewer wrote a paper in 2001 [0] that touches on this and is worth a read (much like all the papers he writes). Hope that helps. Mark ricon2012.com [0] http://www.cs.berkeley.edu/~brewer/Giant.pdf > -- > Dave Brady > > ----- Original Message ----- > From: "Mark Phillips" <m...@basho.com> > To: "Dave Brady" <dbr...@weborama.com> > Cc: riak-users@lists.basho.com > Sent: Tuesday, August 14, 2012 7:34:10 PM GMT +01:00 Amsterdam / Berlin / > Bern / Rome / Stockholm / Vienna > Subject: Re: eLevelDB max_open_files in 1.2.0 > > Hi Dave, > > On Sun, Aug 12, 2012 at 3:58 PM, Dave Brady <dbr...@weborama.com> wrote: >> Hello, >> >> First I want to thanks Basho for greatly expanding the documentation on the >> Wiki for configuring/tuning Riak and eLevelDB in 1.2.0! Big improvement >> over 1.1.x. >> >> My question is about max_open_files: here the documentation is confusing to >> me. >> >> It says to allocate one open file per 2 MB, then divide by the number of >> partitions. This is the same formula used in 1.1.x. >> >> It goes on to say that if you manually set this parameter in 1.1.x, to >> divide that value by two for 1.2.0. >> >> Should not the formula for 1.2.0, in that case, read as use one file per 4 >> MB? > > Long story short, the answer is "yes" :) > > With 1.2, 4MB is the advised file size, and you should be running with > no less that 20 files/nodes. I'll take a pass at updating the docs to > make this a bit easier to understand. Thanks for pointing that out. > > Mark > >> >> Thankks! >> >> -- >> Dave Brady >> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
