Alex, The successor to "last_write_wins" is the "write_once" bucket type. You can read about its characteristics and limitations here:
http://docs.basho.com/riak/kv/2.1.4/developing/app-guide/write-once/ <http://docs.basho.com/riak/kv/2.1.4/developing/app-guide/write-once/> This bucket type eliminates the Riak's typical read-before-write operation. Your experience with better performance by reversing the keys suggests to me that this bucket type might be what you need. Also, I would be willing to review your general environment and particularly leveldb's actions. I would need you to run "riak-debug" on one of the servers then post the tar file someplace private such as dropbox. There might be other insights I can share based upon leveldb's actions and your physical server configuration. Matthew > On May 5, 2016, at 8:32 AM, alexc155 <ahcl...@gmail.com> wrote: > > We're using Riak as a simple key value store and we're having write > performance problems which we think is due to the format of our keys which we > can't easily change because they're tied into different parts of the business > and systems. > > > We're not trying to do anything complicated with Riak: No solr, secondary > indexes or map reducing - just simple keys to strings of around 10Kb of JSON. > > > We've got upwards of 3 billion records to store so we've opted for LevelDb as > the backend. > > > It's a 3 node cluster running on 3 dedicated Ubuntu VMs each with 16 cpu > cores and 12GB memory backed by SSDs on a 10Gb network. > > > Using basho bench we know that it's capable of speeds upwards of 5000 rows > per sec when using randomised keys, but the problem comes when we use our > actual data. > > > The keys are formatted using the following pattern: > > > USC~1930~100000000~10000~001 > USC~1930~100000000~10000~002 > USC~1930~100000000~10000~003 > USC~1930~100000000~10000~004 > > Most of the long key stays the same with numbers at the end going up. (The > "~" are changeable - we can set them to whatever character. They're just > delimiters in the keys) > > > Using these keys, write performance is a tenth of the speed at 400 rows per > sec. > > > We don't need to worry about different versions of the data so we've set the > following in our bucket type: > > > "allow_mult": false > "last_write_wins": true > "DW": 0 > "n_val": 2 > "w": 1 > "r": 1 > "basic_quorum": false > > On the riak servers we've set the ulimit to: > > > riak soft nofile 32768 > riak hard nofile 65536 > > and other settings like this: > > > ring_size = 128 > protobuf.backlog = 1024 > anti_entropy = passive > > We're using the v2 .net client from basho to do the putting which runs in an > API on 3 machines. > > > We've checked all the usual bottlenecks: CPU, memory, network IO, disk IO and > throttles on the Riak servers and windows API servers. > > > As a kicker, if we reverse the keys e.g. > > > 100~00001~000000001~0391~CSU > speed goes up to over 3000 rows, but that's a dirty kludge. > > > Can anyone explain why Riak doesn't like sequential alpha-numeric keys and > what we can change to improve performance? > > > Thanks! > > > View this message in context: How to increase Riak write performance for > sequential alpha-numeric keys > <http://riak-users.197444.n3.nabble.com/How-to-increase-Riak-write-performance-for-sequential-alpha-numeric-keys-tp4034219.html> > Sent from the Riak Users mailing list archive > <http://riak-users.197444.n3.nabble.com/> at Nabble.com. > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
