Re: Load Balancing With Riak Ruby Client
Hey, In my Chef recipes for AppCloud (Engine Yard's PaaS Product) I actually configured haproxy to listen on 8098 on the application instances and redirect to all the riak nodes in a roundrobin fashion. (Had httpchk for /ping to ensure the node is up) In my own testing with basho_bench this seemed to work, I'm unsure of what drawbacks there would be because I could not find any other then HTTP was slower then PBC. I did find find that Protobuffers did not roundrobin correctly with haproxy using tcp mode... darn :( So if we're just speaking HTTP could totally use HAProxy or a Hardware Load balancer to spread out the load. Scott -- Scott M. Likens Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, May 27, 2011 at 12:40 PM, Sean Cribbs wrote: > This is one thing I desperately want to refactor. The Ruby client still > contains some things that reflect my earlier, less astute understanding of > how a Riak client should behave and doesn't include obvious things like > retrying requests (possibly on other nodes), conflict resolution strategies, > and mutators. > > In the past, I have recommended that users put a lightweight load-balancer > (e.g. haproxy, pound) between their application and Riak, and simply have the > app connect to the locally-running instance of the LB. I realize this is not > a great solution, but it also avoids a little NIH for now. > > Sean Cribbs mailto:s...@basho.com)> > Developer Advocate > Basho Technologies, Inc. > http://basho.com/ > > On May 27, 2011, at 3:20 PM, Keith Bennett wrote: > > > Hi, all. If I have several riak servers on a cluster, and want to > > distribute load fairly evenly, and am using the Ruby Riak client, what is > > the best way to balance load? > > > > With the HTTP interface, I can randomize the choice of host for a request. > > How would I do the same with the ruby client? Would I create a Riak::Client > > for each host, and then just randomize the selection of those for a given > > call? Do the clients contain any state that would make this a bad idea? Or > > is there a better way to do this? > > > > Thanks, > > Keith > > > > > > ___ > > riak-users mailing list > > riak-users@lists.basho.com (mailto:riak-users@lists.basho.com) > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > ___ > riak-users mailing list > riak-users@lists.basho.com (mailto:riak-users@lists.basho.com) > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > !DSPAM:4ddffdaf202681804284693! ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
haproxy and protobuffers working example?
Hey, So I thought I had a working haproxy configuration with protobuffers (It worked once upon a time, but no longer) so I was wondering if anyone had any working examples? irb(main):001:0> client = Riak::Client.new(:protocol => "pbc") => # irb(main):002:0> client.ping Riak::ProtobuffsFailedRequest: Expected success from Riak but received server_error. Unexpected EOF on PBC socket irb(main):008:0> client2=Riak::Client.new(:protocol => "pbc", :host => "10.170.121.116") => # irb(main):009:0> client2.ping => true If I remove all the servers except for 1 connects but still doesn't really work, below is my haproxy configuration I hope it helps. listen riak_pbc :8087 mode tcp server app-0 ip-10-170-121-116.us-west-1.compute.internal:8087 check inter 5000 fastinter 1000 fall 1 weight 50 server app-1 ip-10-171-43-226.us-west-1.compute.internal:8087 check inter 5000 fastinter 1000 fall 1 weight 50 server app-2 ip-10-170-142-202.us-west-1.compute.internal:8087 check inter 5000 fastinter 1000 fall 1 weight 50 Actual full haproxy erb chef template can be found here https://github.com/damm/ey-riak/blob/master/templates/default/haproxy.cfg.erb (if you see anything I should turn off for http please let me know and be gentle) Thanks! ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: haproxy and protobuffers working example?
Difference is good and after taking a whirl it seems to have fixed it so thank you very much :) -- Scott M. Likens Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, May 30, 2011 at 4:55 PM, Bob Feldbauer wrote: > Your haproxy.cfg looks fine, but since you asked, here's the my > haproxy.cfg for Riak use: > > global > user haproxy > group haproxy > daemon > maxconn 10 > > defaults > log global > mode tcp > option tcplog > option dontlognull > balance leastconn > clitimeout 6 > srvtimeout 6 > contimeout 5000 > retries 3 > option redispatch > option contstats > stats enable > > listen riak 192.168.1.1:8087 > server riak1 riak1:8087 weight 1 maxconn 1000 > server riak2 riak2:8087 weight 1 maxconn 1000 > server riak3 riak3:8087 weight 1 maxconn 1000 > > > Good luck! > > - Bob Feldbauer > > On 5/30/2011 7:11 PM, Scott M. Likens wrote: > > Hey, > > > > So I thought I had a working haproxy configuration with protobuffers (It > > worked once upon a time, but no longer) so I was wondering if anyone had > > any working examples? > > > > > > irb(main):001:0> client = Riak::Client.new(:protocol => "pbc") > > => # > > irb(main):002:0> client.ping > > Riak::ProtobuffsFailedRequest: Expected success from Riak but received > > server_error. Unexpected EOF on PBC socket > > > > > > > > irb(main):008:0> client2=Riak::Client.new(:protocol => "pbc", :host => > > "10.170.121.116") > > => # > > irb(main):009:0> client2.ping > > => true > > > > > > If I remove all the servers except for 1 connects but still doesn't really > > work, below is my haproxy configuration I hope it helps. > > > > listen riak_pbc :8087 > > mode tcp > > server app-0 ip-10-170-121-116.us-west-1.compute.internal:8087 check inter > > 5000 fastinter 1000 fall 1 weight 50 > > server app-1 ip-10-171-43-226.us-west-1.compute.internal:8087 check inter > > 5000 fastinter 1000 fall 1 weight 50 > > server app-2 ip-10-170-142-202.us-west-1.compute.internal:8087 check inter > > 5000 fastinter 1000 fall 1 weight 50 > > > > Actual full haproxy erb chef template can be found here > > https://github.com/damm/ey-riak/blob/master/templates/default/haproxy.cfg.erb > > (if you see anything I should turn off for http please let me know and be > > gentle) > > !DSPAM:4de42eb0180691550818619! ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
