Why not use load balancer on top of Riak cluster, independently from clients? If load balancing is sophisticated enough, it can even do a greater job than just uniformly spreading requests between machines.
Consider the following situation. You happen to send several requests for a huge piece of data (each can be just a simple get, or even a complicated map-reduce query), but the rest of the nodes, or just some of them, receive much smaller queries. What an external load balancer can do is gathering some basic stats from each machine (like load average, memory and IO consumption, number of rps to Riak and so on) and balancing the requests according to this data. Another benefit from this approach is that you can easily add any other Riak client (Python, Ruby, whatever else) on top of it, and load will be still nicely distributed between the machines. On Mon, Sep 24, 2012 at 11:38 AM, Yuri Lukyanov <y.sn...@gmail.com> wrote: > Hi, > > I'm trying to use poolboy to organize a pool of connections to riak > nodes with protobuf client. > > This has come from the suggestion here: > > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2012-September/009346.html > > The quote: > "Using Poolboy is convenient because it comes as a dependency of > riak_core. > > If you use Poolboy, you'll have to modify riakc_pb_socket slightly to > account for the way poolboy initializes connections (add a start_link/1), > or create a simple module to pass the initialization from poolboy to > riakc_pb_socket." > > What I'm pondering about is what would be a convenient way to start > riakc_pb_socket clients in a pool. > For example, if I have 10 riak nodes in a cluster and want to open a > single connection to each of them (using riakc_pb_socket), I need one > poolboy worker per one riakc_pb_socket instance. In this case, I need > a host/port of a single riak node for each > riakc_pb_socket:start_link(). So I use WorkerArgs in > poolboy:start_link/3 to pass a list of riak nodes. And here comes a > confusion. poolboy passes WorkerArgs to each of its workers, so each > worker gets the list of riak nodes. Now I need to somehow choose a > node, maybe like proposed here: > https://github.com/basho/riak-erlang-client/pull/45/files > But I don't think it's a good solution. > > Maybe my idea of using riakc_pb_socket together with poolboy is not > convenient? Maybe I should start a pool per each riak node? Maybe > poolboy is not that convenient in this case as it seems? > > Would you share your experiences and thoughts on the subject? > > Thank you. > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > -- Best regards, Dmitry Demeshchuk
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
