If using HTTP, make sure you're using keep-alives. That will be a gigantic speed boost.
The protocol buffer API is much faster if you're client language supports it. Mark Steele Bering Media Inc. On Thu, Apr 7, 2011 at 10:58 PM, matthew hawthorne <mhawtho...@gmail.com>wrote: > Hi Gui, > > I recently pushed 70 million records of size 1K each into a 5-node > Riak cluster (which was replicating to another 5-node cluster) at > around 1000 writes/second using basho_bench and the REST interface. I > probably could have pushed it further, but I wanted to confirm that it > could maintain the load for the entire data set, which it did. > > My point being that your speed-limit of 200 writes/second is likely > specific to your configuration. > > I wonder: > 1) what's your average write latency? > 2) how big is your connection pool? > > Because it's possible that you don't have enough connections available > to handle your desired load. > > -matt > > > On Thu, Apr 7, 2011 at 6:01 PM, Gui Pinto <gpi...@chitika.com> wrote: > > Hey guys, > > I'm attempting to importing 300M+ objects into a Riak cluster, but have > > quickly reached the REST API's speed-limit at 200-store()'s per second.. > > At the rate of 200/s, I'm looking at 20-days to import this data set! > That > > can't be the fastest method to do this.. > > > > Any recommendations? > > > > Thanks! > > Gui Pinto > > > > _______________________________________________ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
