It turned out this was caused by my using the wrong port in my code.
I had two ways of initiating the connection to riak in my python code. One
using the HTTP interface, and one using the protocol buffers interface, and
the port being used for protocol buffers was incorrectly hitting the system
ri
More info, in case it helps someone help me figure out what has gone wrong.
I suspect I have something misconfigured, or not configured, that is
affecting the mapreduce js vms.
I followed the instructions on this thread:
http://riak-users.197444.n3.nabble.com/preflist-exhausted-with-map-reduce-and
tting the same
"preflist_exhausted" error. The number of records have grown in Oct, Nov, Jan,
Feb etc. How do you deal with the data growth?
Thanks
Mohit
From: Sean Cribbs [s...@basho.com]
Sent: Tuesday, April 24, 2012 11:58 AM
To: Sati, Moh
, 2012 11:58 AM
To: Sati, Mohit
Cc: riak-users@lists.basho.com
Subject: Re: preflist_exhausted error
Mohit,
Be aware that key filters are just a thin layer on top of full-bucket key
listings. You'd be better off storing the field you want to filter in a
secondary index, which more effici
Sati,
Check, if you see the following error in your log file:
crasher:
initial call: riak_kv_js_vm:init/1
pid: <0.404.0>
registered_name: []
exception exit: {{badmatch,{error
,enoent}},[{riak_kv_js_vm,load_mapred_builtins,1},{js_driver,new,3},{riak_kv_js_vm,init,1},{gen_server,init
Mohit,
Be aware that key filters are just a thin layer on top of full-bucket key
listings. You'd be better off storing the field you want to filter in a
secondary index, which more efficiently supports range queries (note that
only the LevelDB storage engine currently supports secondary indexes).
Thanks for the assistance, Jon!
-Matthew
On Wed, Apr 4, 2012 at 13:00, Jon Meredith wrote:
> After some investigating we tracked this down to a missing mapred_builtins.js
> file on the system. There were error messages in the console.log
>
> The issue https://github.com/basho/riak_pipe/issu
After some investigating we tracked this down to a missing mapred_builtins.js
file on the system. There were error messages in the console.log
The issue https://github.com/basho/riak_pipe/issues/44 identified on this
thread still stands for some of the preflist_exhausted issues, this was
just an
Jon,
I even restarted all of the nodes in the cluster and it's still zero.
-Matthew
On Tue, Apr 3, 2012 at 16:56, Jon Meredith wrote:
> That's very interesting, you don't have any Javascript VMs running. Here's
> an example on a single node system after restart - I've configured it with
> 2
That's very interesting, you don't have any Javascript VMs running. Here's
an example on a single node system after restart - I've configured it with
2 map VMs and 3 reduce VMs just to get easy numbers.
(dev1@127.0.0.1)1>
rpc:multicall(riak_kv_js_manager,pool_size,[riak_kv_js_map]).
{[2],[]}
(dev1
Jon,
We dont have any MR jobs running at all :)) Here you go:
1>rpc:multicall(supervisor, count_children, [riak_pipe_builder_sup]).
{[[{specs,1},{active,0},{supervisors,0},{workers,0}],
[{specs,1},{active,0},{supervisors,0},{workers,0}],
[{specs,1},{active,0},{supervisors,0},{workers,0}]],
[
Hi Matthew,
Thanks for the logs. Do you know how many map/reduce jobs you run
concurrently? Is this a load-based issue or do you run into problems?
Running this from the erlang console should give an idea of how many pipes,
fittings and JS VMs you are running concurrently ('riak attach' to star
Very strange.
Are you seeing the
16:40:50.037 [notice] JS call failed: All VMs are busy.
message in your console.log files on any of the nodes in your cluster?
Jon
On Mon, Apr 2, 2012 at 11:48 AM, Matthew Tovbin wrote:
> Jon,
>
> Thanks for the reply. I tried increasing the 'map_js_vm_count'
Jon,
Thanks for the reply. I tried increasing the 'map_js_vm_count' and
'reduce_js_vm_count'
on the cluster to the values you suggested 24/18 and even larger ones 32/24
and 128/128, but with no success - I still get the same error in the
console log of the node.
What are we missing?!
-Matthew
Hi list,
I spend some time looking at this issue today. I suspect it is due to all
the Javascript VMs being busy inside Riak due to many parallel MapReduce
jobs.
If you're seeing this issue please check for this message in the
console.log file
16:40:50.037 [notice] JS call failed: All VMs are b
Guys,
Any updates on the issue?!
-Matthew
On Tue, Mar 13, 2012 at 18:29, Matthew Tovbin wrote:
> Here is a log from one of the servers:
>
> ==> /mnt/dataraid/log/crash.log <==
> 2012-03-13 18:24:44 =CRASH REPORT
> crasher:
> initial call: riak_pipe_builder:init/1
> pid: <0.2094
Here is a log from one of the servers:
==> /mnt/dataraid/log/crash.log <==
2012-03-13 18:24:44 =CRASH REPORT
crasher:
initial call: riak_pipe_builder:init/1
pid: <0.20949.24>
registered_name: []
exception exit: {sink_died,shutdown}
in function gen_fsm:terminate/7
Hi,
I got the same problem today on 1.1.0.
As suggested, I updated all the nodes to 1.1.1. The error remains the same:
{ stack: [Getter/Setter],
arguments: undefined,
type: undefined,
message: 'HTTP error 500:
{"phase":0,"error":"[preflist_exhausted]","input":"{ok,{r_object ."}}}
-Ma
Hi Arne,
Could you try with 1.1.1 please. We've fixed a number of bugs with how
MapReduce handles errors that could explain some of what you are seeing.
Jon
On Tue, Mar 13, 2012 at 12:55 AM, Arne Claassen wrote:
> I'm running 1.1.0 built from source on Centos 6. I set up two nodes on
> localho
19 matches
Mail list logo
