CRDTs are all about mutation, why would you use a CRDT for immutable data? I think write-once is what you need.
On 5 May 2016, at 14:14, Sanket Agrawal <[email protected]> wrote: > Two questions about write-once bucket: > - is it useful from performance-perspective for immutable CRDTs as well? > - If the objects are immutable, is map CRDT generally performant, compared to > equivalent JSON with write-once property? Let us say map has 1-9 registers > with about 2KB size total in string/number. > > It seems to me there is no way to enforce key check before writing CRDTs (to > enforce true immutability by making sure the new key doesn't exist already) > as it is for key-value. We could tell Riak not to insert a key if it already > exists for key-value objects. I don't know of any such mechanism for CRDTs. > > I have immutable CRDTs with map type, allow_mult true, and lww false. I > looked into "write-once" property before but couldn't figure out how to truly > enforce it for CRDT. > > > > > On Thu, May 5, 2016 at 8:51 AM, Matthew Von-Maszewski <[email protected]> > wrote: > Alex, > > The successor to "last_write_wins" is the "write_once" bucket type. You can > read about its characteristics and limitations here: > > http://docs.basho.com/riak/kv/2.1.4/developing/app-guide/write-once/ > > This bucket type eliminates the Riak's typical read-before-write operation. > Your experience with better performance by reversing the keys suggests to me > that this bucket type might be what you need. > > Also, I would be willing to review your general environment and particularly > leveldb's actions. I would need you to run "riak-debug" on one of the > servers then post the tar file someplace private such as dropbox. There > might be other insights I can share based upon leveldb's actions and your > physical server configuration. > > Matthew > > > >> On May 5, 2016, at 8:32 AM, alexc155 <[email protected]> wrote: >> >> We're using Riak as a simple key value store and we're having write >> performance problems which we think is due to the format of our keys which >> we can't easily change because they're tied into different parts of the >> business and systems. >> >> >> We're not trying to do anything complicated with Riak: No solr, secondary >> indexes or map reducing - just simple keys to strings of around 10Kb of JSON. >> >> >> We've got upwards of 3 billion records to store so we've opted for LevelDb >> as the backend. >> >> >> It's a 3 node cluster running on 3 dedicated Ubuntu VMs each with 16 cpu >> cores and 12GB memory backed by SSDs on a 10Gb network. >> >> >> Using basho bench we know that it's capable of speeds upwards of 5000 rows >> per sec when using randomised keys, but the problem comes when we use our >> actual data. >> >> >> The keys are formatted using the following pattern: >> >> >> USC~1930~100000000~10000~001 >> USC~1930~100000000~10000~002 >> USC~1930~100000000~10000~003 >> USC~1930~100000000~10000~004 >> >> >> Most of the long key stays the same with numbers at the end going up. (The >> "~" are changeable - we can set them to whatever character. They're just >> delimiters in the keys) >> >> >> Using these keys, write performance is a tenth of the speed at 400 rows per >> sec. >> >> >> We don't need to worry about different versions of the data so we've set the >> following in our bucket type: >> >> >> "allow_mult": false >> "last_write_wins": true >> "DW": 0 >> "n_val": 2 >> "w": 1 >> "r": 1 >> "basic_quorum": false >> >> >> On the riak servers we've set the ulimit to: >> >> >> riak soft nofile 32768 >> riak hard nofile 65536 >> >> >> and other settings like this: >> >> >> ring_size = 128 >> protobuf.backlog = 1024 >> anti_entropy = passive >> >> >> We're using the v2 .net client from basho to do the putting which runs in an >> API on 3 machines. >> >> >> We've checked all the usual bottlenecks: CPU, memory, network IO, disk IO >> and throttles on the Riak servers and windows API servers. >> >> >> As a kicker, if we reverse the keys e.g. >> >> >> 100~00001~000000001~0391~CSU >> speed goes up to over 3000 rows, but that's a dirty kludge. >> >> >> Can anyone explain why Riak doesn't like sequential alpha-numeric keys and >> what we can change to improve performance? >> >> >> Thanks! >> >> >> View this message in context: How to increase Riak write performance for >> sequential alpha-numeric keys >> Sent from the Riak Users mailing list archive at Nabble.com. >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
