My understanding is that it's not riak which ultimately limits the size of the cluster but erlang distribution protocol. By default erlang nodes are fully connected, so if you have 1000 nodes you have about 999 outgoing connections from each node, and with the net tick, they start to receive a lot of chatter.
I believe that scalaris may be engineered for larger growth by not using erlang distribution protocol but their own, and only have a few outgoing connections per node. I'm also interested in what basho has to say. -Anthony On Wed, Apr 13, 2011 at 12:41:09PM -0700, Greg Nelson wrote: > The largest known riak cluster != the largest possible riak cluster. ;-) > > The inter node communication of the cluster depends on the data set and usage > pattern, doesn't it? Or is there some constant overhead that tops out at a > few hundred nodes? I should point out that we'll have big data, but not a > huge number of keys. > > The number of vnodes in the cluster should be equal to the ring_creation_size > under normal circumstances, shouldn't it? So when I have a one node cluster, > that node is running ring_creation_size vnodes... File descriptors probably > isn't a problem -- these machines won't be doing anything else, and the > limits are set to 65536. > > Thinking about the internode communication you mentioned, that's probably > where the resource hog is.. socket buffers, etc. > > Anyway, I'd also love to hear more from basho. :) > On Wednesday, April 13, 2011 at 12:33 PM, sicul...@gmail.com wrote: > > Ill just chime in and say that this is not practical for a few reasons. The > > largest known riak cluster has like 50 or 60 nodes. Afaik, inter node > > communication of erlang clusters top out at a few hundred nodes. I'm also > > under the impression that each physical node has to have enough file > > descriptors to accommodate every virtual node in the cluster. > > > > I'd love to hear more from basho. > > > > -alexander > > > > > > Sent from my Verizon Wireless BlackBerry > > > > -----Original Message----- > > From: Greg Nelson <gro...@dropcam.com> > > Sender: riak-users-boun...@lists.basho.com > > Date: Wed, 13 Apr 2011 12:13:34 > > To: <riak-users@lists.basho.com> > > Subject: Large ring_creation_size > > > > _______________________________________________ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com -- ------------------------------------------------------------------------ Anthony Molinaro <antho...@alumni.caltech.edu> _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
