Two questions about write-once bucket: - is it useful from performance-perspective for immutable CRDTs as well? - If the objects are immutable, is map CRDT generally performant, compared to equivalent JSON with write-once property? Let us say map has 1-9 registers with about 2KB size total in string/number.
It seems to me there is no way to enforce key check before writing CRDTs (to enforce true immutability by making sure the new key doesn't exist already) as it is for key-value. We could tell Riak not to insert a key if it already exists for key-value objects. I don't know of any such mechanism for CRDTs. I have immutable CRDTs with map type, allow_mult true, and lww false. I looked into "write-once" property before but couldn't figure out how to truly enforce it for CRDT. On Thu, May 5, 2016 at 8:51 AM, Matthew Von-Maszewski <matth...@basho.com> wrote: > Alex, > > The successor to "last_write_wins" is the "write_once" bucket type. You > can read about its characteristics and limitations here: > > http://docs.basho.com/riak/kv/2.1.4/developing/app-guide/write-once/ > > This bucket type eliminates the Riak's typical read-before-write > operation. Your experience with better performance by reversing the keys > suggests to me that this bucket type might be what you need. > > Also, I would be willing to review your general environment and > particularly leveldb's actions. I would need you to run "riak-debug" on > one of the servers then post the tar file someplace private such as > dropbox. There might be other insights I can share based upon leveldb's > actions and your physical server configuration. > > Matthew > > > > On May 5, 2016, at 8:32 AM, alexc155 <ahcl...@gmail.com> wrote: > > We're using Riak as a simple key value store and we're having write > performance problems which we think is due to the format of our keys which > we can't easily change because they're tied into different parts of the > business and systems. > > We're not trying to do anything complicated with Riak: No solr, secondary > indexes or map reducing - just simple keys to strings of around 10Kb of > JSON. > > We've got upwards of 3 billion records to store so we've opted for LevelDb > as the backend. > > It's a 3 node cluster running on 3 dedicated Ubuntu VMs each with 16 cpu > cores and 12GB memory backed by SSDs on a 10Gb network. > > Using basho bench we know that it's capable of speeds upwards of *5000 > rows per sec* when using randomised keys, but the problem comes when we > use our actual data. > > The keys are formatted using the following pattern: > > USC~1930~100000000~10000~001 > USC~1930~100000000~10000~002 > USC~1930~100000000~10000~003 > USC~1930~100000000~10000~004 > > > Most of the long key stays the same with numbers at the end going up. (The > "~" are changeable - we can set them to whatever character. They're just > delimiters in the keys) > > Using these keys, write performance is a tenth of the speed at *400 rows > per sec*. > > We don't need to worry about different versions of the data so we've set > the following in our bucket type: > > "allow_mult": false > "last_write_wins": true > "DW": 0 > "n_val": 2 > "w": 1 > "r": 1 > "basic_quorum": false > > > On the riak servers we've set the ulimit to: > > riak soft nofile 32768 > riak hard nofile 65536 > > > and other settings like this: > > ring_size = 128 > protobuf.backlog = 1024 > anti_entropy = passive > > > We're using the v2 .net client from basho to do the putting which runs in > an API on 3 machines. > > We've checked all the usual bottlenecks: CPU, memory, network IO, disk IO > and throttles on the Riak servers and windows API servers. > > As a kicker, if we reverse the keys e.g. > > 100~00001~000000001~0391~CSU > > speed goes up to over *3000 rows*, but that's a dirty kludge. > > Can anyone explain why Riak doesn't like sequential alpha-numeric keys and > what we can change to improve performance? > > Thanks! > > ------------------------------ > View this message in context: How to increase Riak write performance for > sequential alpha-numeric keys > <http://riak-users.197444.n3.nabble.com/How-to-increase-Riak-write-performance-for-sequential-alpha-numeric-keys-tp4034219.html> > Sent from the Riak Users mailing list archive > <http://riak-users.197444.n3.nabble.com/> at Nabble.com. > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
