More info, in case it helps someone help me figure out what has gone wrong. I suspect I have something misconfigured, or not configured, that is affecting the mapreduce js vms.
I followed the instructions on this thread: http://riak-users.197444.n3.nabble.com/preflist-exhausted-with-map-reduce-and-riak-js-td3821516.html In that thread, the guys problem turned out to be a missing mapred_builtins.js file. I found mine /usr/lib/riak/lib/riak_kv-1.2.0/priv so I don't think my problem is the same. About using the riak console to find out how many map reduce vms are available: (riak@127.0.0.1)5> rpc:multicall(supervisor, count_children, [riak_pipe_builder_sup]). {[[{specs,1},{active,0},{supervisors,0},{workers,0}]],[]} (riak@127.0.0.1)6> rpc:multicall(supervisor, count_children, [riak_pipe_fitting_sup]). {[[{specs,1},{active,0},{supervisors,0},{workers,0}]],[]} (riak@127.0.0.1)7> rpc:multicall(riak_kv_js_manager,pool_size,[riak_kv_js_map]). {"\b",[]} (riak@127.0.0.1)9> rpc:multicall(riak_kv_js_manager,pool_size,[riak_kv_js_reduce]). {[6],[]} May be worth noting that those commands product the same weird output whether my app.config settings are: {map_js_vm_count, 32 }, {reduce_js_vm_count, 32 }, {hook_js_vm_count, 16 }, Or the outlandishly high: {map_js_vm_count, 4096 }, {reduce_js_vm_count, 4096 }, {hook_js_vm_count, 256 }, and look nothing like the Mathew Tobvin's output: 3>rpc:multicall(riak_kv_js_manager,pool_size,[riak_kv_js_map]). {[0,0,0],[]} 4>rpc:multicall(riak_kv_js_manager,pool_size,[riak_kv_js_reduce]). {[0,0,0],[]} Also, the output of those commands is the same after running a mapreduce job. Once in a while, usually after a fresh riak restart and a bit of time has passed since the last preflist_exhausted error, one of my mapreduce jobs will actually complete successfully. At a mere 10000 keys in one bucket and exactly 1 user, I do not understand at all what is wrong in my installation. Does anyone have any suggestions? Darrel On Tue, Sep 18, 2012 at 9:27 AM, Darrel Grant <darrel...@gmail.com> wrote: > I have about 10000 items in a bucket, and am getting this error a lot. > > My app.config settings are already quite a lot higher than any values I've > come across while googling this issue: > {map_js_vm_count, 4096 }, > {reduce_js_vm_count, 4096 }, > {hook_js_vm_count, 256 }, > {js_max_vm_mem, 256}, > > And just steadily increasing them doesn't seem good, because this bucket > could easily have millions of keys in it in a bit of time. > > Interestingly, if I stop and start riak, I can get the results I expect > from the mapreduce, but if I then run it again, I get the > preflist_exhausted error. Is there some setting I am missing, which frees > up workers after a run, or perhaps causes them to be re-used in some way? >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
