Yes, we have have jumbo frames on. Kernel and systems are in a good state. I've got the settings from the Linux perf tuning guide on the riak docs. Testing with iperf(1) shows our nodes can saturate the 10G interfaces without breaking a sweat - as long as the packets are large enough...
1) http://iperf.sourceforge.net/ On Sat, Oct 26, 2013 at 11:36 AM, Sean Cribbs <s...@basho.com> wrote: > Chris, > > Are you using jumbo frames on your network? (MTU 9000 or w/e) That's just > one aspect, but we've also found there are many other kernel/TCP-stack > tunings that need to be done to get Riak to push the limits of the network. > > > On Fri, Oct 25, 2013 at 9:10 PM, Chris Read <chris.r...@gmail.com> wrote: > >> Greetings all... >> >> Context: We're running Riak 1.4.2 on Ubuntu 12.04 using the Basho build >> .deb package. Out object size varies between 1K and 512K >> >> I've been having some trouble trying to figure out why handoffs between >> our nodes takes so long and never pushes the network card over 2.5 Gb/s. >> Testing communication between nodes using iperf shows that the >> configuration we have can sustain 10G. >> >> Further investigation shows that communication between nodes never uses >> messages > 4k which gets the network cards to top out on packages per >> second before they reach the top bits per second. >> >> Possibly related: I see exactly the same problem when using Basho Bench >> and the pb driver - I can never get the client to go over 2.5Gb/s. This is >> pushing me to suspect erlang vm or something as the problem. >> >> Anyone seen this before? Is there a magic "use bigger messages" flag >> hidden somewhere? >> >> Thanks, >> >> Chris >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > -- > Sean Cribbs <s...@basho.com> > Software Engineer > Basho Technologies, Inc. > http://basho.com/ >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
