I think the slowness is coming from the older list keys implementation in
0.12.1, list keys has been changed in the tip version of Riak and is quite a
bit faster now. In addition there have been a lot of improvements to the
Javascript map reduce implementation that should help the speed of your query.
For the time being you will need to run Riak tip to get access to these
enhancements.
Grant Schofield
Developer Advocate
Basho Technologies, Inc.
On Sep 16, 2010, at 5:17 PM, Nils Petersohn wrote:
> ok, my ring seems ok now.
> what i did was to change the rel/vars/dev[1,2,3]_vars.config file.
> in there i was just replacing the ips...
> this reip thing did not really work out ...
>
> here is my riak ring now:
> (d...@192.168.0.100)1> riak_core_ring_manager:get_my_ring().
> {ok,{chstate,'d...@192.168.0.100',
> [{'d...@192.168.0.107',{65,63451889794}},
> {'d...@192.168.0.105',{13,63451889512}},
> {'d...@192.168.0.100',{104,63451889512}},
> {'d...@192.168.0.105',{49,63451889512}},
> {'d...@192.168.0.100',{32,63451889009}},
> {'d...@192.168.0.105',{94,63451889253}},
> {'d...@192.168.0.107',{9,63451889769}},
> {'d...@192.168.0.100',{97,63451889494}}],
> {64,
> [{0,'d...@192.168.0.100'},
> {22835963083295358096932575511191922182123945984,
> 'd...@192.168.0.105'},
> {45671926166590716193865151022383844364247891968,
> 'd...@192.168.0.107'},
> {68507889249886074290797726533575766546371837952,
> 'd...@192.168.0.100'},
> {91343852333181432387730302044767688728495783936,
> 'd...@192.168.0.105'},
> {114179815416476790484662877555959610910619729920,
> 'd...@192.168.0.107'},
> {137015778499772148581595453067151533092743675904,
> 'd...@192.168.0.100'},
> {159851741583067506678528028578343455274867621888,
> 'd...@192.168.0.105'},
> {182687704666362864775460604089535377456991567872,
> 'd...@192.168.0.100'},
> {205523667749658222872393179600727299639115513856,
> 'd...@192.168.0.105'},
> {228359630832953580969325755111919221821239459840,
> 'd...@192.168.0.107'},
> {251195593916248939066258330623111144003363405824,
> 'd...@192.168.0.100'},
> {274031556999544297163190906134303066185487351808,
> 'd...@192.168.0.105'},
> {296867520082839655260123481645494988367611297792,
> 'd...@192.168.0.107'},
> {319703483166135013357056057156686910549735243776,
> 'd...@192.168.0.100'},
> {342539446249430371453988632667878832731859189760,
> 'd...@192.168.0.105'},
> {365375409332725729550921208179070754913983135744,
> 'd...@192.168.0.100'},
> {388211372416021087647853783690262677096107081728,
> 'd...@192.168.0.105'},
> {411047335499316445744786359201454599278231027712,
> 'd...@192.168.0.107'},
> {433883298582611803841718934712646521460354973696,...},
> {...}|...]},
> {dict,0,16,16,8,80,48,
> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],...},
> {{[],[],[],[],[],[],[],[],[],[],[],[],...}}}}}
> (d...@192.168.0.100)2>
>
> i am using 0.12.1 on my mac and 0.12 on both vms. i have now a set of 100.000
> entrys like this (just for testing):
> {"id":"42164", "actionTime":"2007-05-11 17:08:55", "action":"some action",
> "res":"7024", "user":"5", "client":"2787"}
>
>
> and my mr job looks like this (just for testing):
> {"inputs":"actionbucket",
> "query":[
> {"map":{"language":"javascript", "source":
> "function(values, keyData, arg) {
>
> var value = Riak.mapValuesJson(values)[0];
> if(value.reservation == '4084'){
> return [value];
> }
> return [];
> }","keep":true}}
> ],"timeout": 900000
> }
>
>
> the beam instances are all showing on "top" now, and there is some traffic
> going back and forth. (~200kb / s)
>
> but this job takes like 1:30 min.
>
> i know that this is not really comparable with a mysql query because you can
> do more calculations in the mr job to produce much more special results and
> the mr job has a ~linear "worktime"... but ~1:30 min is still pretty bad ....
>
> is there any way to do much better ?
>
> best regards
> nils
>
> On Sep 16, 2010, at 7:08 PM, Grant Schofield wrote:
>
>>
>> On Sep 15, 2010, at 2:40 PM, Nils Petersohn wrote:
>>
>>> hello,
>>>
>>> i was setting up 9 riak instances:
>>>
>>> three on my mac with the appropriate app config
>>> and six with two virtual machines on a different computer.
>>>
>>> all 8 joined the d...@192.168.1.20
>>> and the join request was sent.
>>>
>>> after setting this up:
>>> i wanted to put data with the java client on d...@192.168.1.20 than i got a
>>> timeout ?!?
>>>
>>
>> I am curious if you started this node and then changed its name in the
>> config file? Errors like this can happen if you don't riak-admin reip the
>> node, also the ring file would be wrong and this could lead to some of the
>> other errors you saw below. One thing you may want to look at is the state
>> of your ring from the Riak console using
>> riak_core_ring_manager:get_my_ring(). That might show any problems with the
>> ring, feel free to send that along so we can take a look at it.
>>
>>> when i put data on one of the other machines than only this machine was
>>> using cpu time and none of the other ...
>>> if consistent hashing works like expected, than all the machines should
>>> show up on "top"
>>>
>>> when i did a mapreduce job than only this machine was using cpu time and
>>> none of the other ...
>>>
>>> i had "top" running on all of them.
>>>
>>> -------------------------------------------------------
>>> the other problem is:
>>>
>>> when i have 1/2 mio. entrys in one bucket with less than 100 chars for each
>>> entry
>>> and i do a really simple mapreduce job, than it takes forever (15 minutes
>>> ...)
>>> while sql uses .005 secons....
>>>
>>> i know that doing a mr on a complete bucket, than it takes very long if i
>>> don't secify keys in the bucket. but how should i know which keys to use ...
>>
>> What version of Riak are you using? There has been a fair amount of
>> improvement to the map reduce system as well as list keys. Are the map
>> reduce jobs you are running javascript?
>>
>>> ------------------------------------------------------
>>>
>>> if i put stuff in one bucket and add a machine with the join request, how
>>> can i rebalance the bucket???? so that the other machine is taking some
>>> values too.
>>
>> This happens automatically. When the new node joins the cluster you should
>> see handoff messages in the erlang.log.X log file. Rebalancing is handled
>> by the cluster and shouldn't be done manually.
>>
>> Grant Schofield
>> Developer Advocate
>> Basho Technologies, Inc.
>>
>>
>>>
>>> ------------------------------------------------------
>>>
>>> i don't understand these issues/behaviors (timeout, 15min. etc.,
>>> rebalancing), maybe i was setting the one of the three params incorrect ? i
>>> left everything to the default settings.
>>>
>>> thx in advance for any hints...
>>>
>>> nils
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users@lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
> Nils M. Petersohn
> xing.com/profile/Nils_Petersohn
> blog.srvme.de
> twitter.com/snackycracky
> facebook.com/nils.petersohn
> myspace.com/electrash
>
> p...@berlin.de
> 0049 (0)151 40 511 351
> skype: nilz_berlin
>
> Ebertystr. 47
> 10249 Berlin
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
