Hi, Indeed you're using very big keys. If you can't change the keys, then yes you'll have to use leveldb. However I wonder why you need keys that long :)
On 10 July 2013 13:04, Edgar Veiga <edgarmve...@gmail.com> wrote: > Hi Damien, > > Well let's dive into this a little bit. > > I told you guys that bitcask was not an option due to a bad past > experiencie with couchbase (sorry, in the previous post I wrote couchdb), > that uses the same architecture as bitcask, keys in memory and values in > disk. > > We started the migration to couchbase, and were already using 3 physical > nodes and only had imported 5% of the real data! That was one of the main > reasons for choosing a solution like riak + leveldb, to delete the keys fit > in memory bottleneck.. > > Now, here's a tipical key (It's large :D, x=letter and 0=numbers ): > > xxxxxxx_xxxxxxxx_xxxxxxxx_00000xxxxx_0000000000_00000000000_0000000xxxxxx > > We are using the php serialize native function! > > Best regards > > > > On 10 July 2013 11:43, damien krotkine <dkrotk...@gmail.com> wrote: > >> >> >> >> On 10 July 2013 11:03, Edgar Veiga <edgarmve...@gmail.com> wrote: >> >>> Hi Guido. >>> >>> Thanks for your answer! >>> >>> Bitcask it's not an option due to the amount of ram needed.. We would >>> need a lot more of physical nodes so more money spent... >>> >> >> Why is it not an option? >> >> If you use Bitcask, then each node needs to store its keys in memory. >> It's usually not a lot. In a precedent email I asked you the average lenght >> of *keys*, but you gave us the average length of *values* :) >> >> We have 1 billion keys and fits on a 5 nodes Ring. ( check out >> http://docs.basho.com/riak/1.2.0/references/appendices/Bitcask-Capacity-Planning/). >> Our bucket names are 1 letter, our keys are 10 chars long. >> >> What does a typical key look like ? Also, what are you using to serialize >> your php objects? Maybe you could paste a typical value somewhere as well >> >> Damien >> >> >>> >>> Instead we're using less machines with SSD disks to improve elevelDB >>> performance. >>> >>> Best regards >>> >>> >>> >>> On 10 July 2013 09:58, Guido Medina <guido.med...@temetra.com> wrote: >>> >>>> Well, I rushed my answer before, if you want performance, you probably >>>> want Bitcask, if you want compression then LevelDB, the following links >>>> should help you decide better: >>>> >>>> http://docs.basho.com/riak/1.2.0/tutorials/choosing-a-backend/Bitcask/ >>>> http://docs.basho.com/riak/1.2.0/tutorials/choosing-a-backend/LevelDB/ >>>> >>>> Or multi, use one as default and then the other for specific buckets: >>>> >>>> http://docs.basho.com/riak/1.2.0/tutorials/choosing-a-backend/Multi/ >>>> >>>> HTH, >>>> >>>> Guido. >>>> >>>> >>>> >>>> On 10/07/13 09:53, Guido Medina wrote: >>>> >>>> Then you are better off with Bitcask, that will be the fastest in your >>>> case (no 2i, no searches, no M/R) >>>> >>>> HTH, >>>> >>>> Guido. >>>> >>>> On 10/07/13 09:49, Edgar Veiga wrote: >>>> >>>> Hello all! >>>> >>>> I have a couple of questions that I would like to address all of you >>>> guys, in order to start this migration the best as possible. >>>> >>>> Context: >>>> - I'm responsible for the migration of a pure key/value store that for >>>> now is being stored on memcacheDB. >>>> - We're serializing php objects and storing them. >>>> - The total size occupied it's ~2TB. >>>> >>>> - The idea it's to migrate this data to a riak cluster with elevelDB >>>> backend (starting with 6 nodes, 256 partitions. This thing is scaling very >>>> fast). >>>> - We only need to access the information by key. *We won't need >>>> neither map/reduces, searches or secondary indexes*. It's a pure >>>> key/value store! >>>> >>>> My questions are: >>>> - Do you have any riak fine tunning tip regarding this use case (due to >>>> the fact that we will only use the key/value capabilities of riak)? >>>> - It's expected that those 2TB would be reduced due to the levelDB >>>> compression. Do you think we should compress our objects to on the client? >>>> >>>> Best regards, >>>> Edgar Veiga >>>> >>>> >>>> _______________________________________________ >>>> riak-users mailing >>>> listriak-users@lists.basho.comhttp://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> riak-users mailing list >>>> riak-users@lists.basho.com >>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>>> >>> >>> _______________________________________________ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
