I got the same error on an AWS instance (m1.xlarge) when using the MapReduce
version of list all keys.
Query failed with Riak returned an error. Code '0'. Message:
{"phase":0,"error":"[preflist_exhausted]","input":"{ok,{r_object,<<\"buyseas
ons-products\">>,<<\"00113023\">>,[{r_content,{dict,4,16,16,8,80,48,{[],[],[
],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[[
<<\"content-type\">>,97,112,112,108,105,99,97,116,105,111,110,47,106,115,111
,110],[<<\"X-Riak-VTag\">>,49,54,99,97,90,90,72,106,56,50,100,77,85,75,66,11
4,76,50,109,88,89,109]],[[<<\"index\">>,{<<\"active_bin\">>,<<\"InactiveDisc
ontinued\">>},{<<\"definition_bin\">>,<<\"Costume\">>},{<<\"department_bin\"
>>,<<\"Adult
Costumes\">>},{<<\"...\">>,...}]],...}}},...}],...},...}","type":"forward_pr
eflist","stack":"[]"} - CommunicationError
The 'Department' MapReduce with an AWS instance also returned three 'failed'
phases.
Kevin
From: riak-users [mailto:riak-users-boun...@lists.basho.com] On Behalf Of
Jeremiah Peschka
Sent: Tuesday, February 26, 2013 3:18 PM
To: riak-users
Subject: Re: MapReduce performance problem
Responses inline.
---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop
On Tue, Feb 26, 2013 at 12:26 PM, Kevin Burton <rkevinbur...@charter.net>
wrote:
Right. I know it is not ideal. I have been able to split the VM's into
groups. So 2 of the 4 are running on separate hardware. Anything more I just
get the response 'get real'. That being said I want to get the maximum
performance of the limited resources that I have. I have a separate question
for the group in trying to get basho_bench up and running (I get a long
string of errors). What do you need to know more about my environment to
"understand" it? I am new so I am probably asking the wrong questions so
please tell me what you are missing that might help diagnose the problem.
For troubleshooting any environment it's good to know relevant hardware
details about CPU speed, core count, amount of RAM, disks, network cards,
etc. A working basho_bench benchmark would help, too, because it will
provide an indicator of how your environment will perform with Riak as
opposed to how your business logic performs, as implemented, with Riak.
For virtualization, it's also important to know how many other guests are on
the host, whether there are any CPU, memory, or network reservations in
place, and which version of virtualization you're running. Virtualization
makes performance tuning more complex, but not impossible.
I agree. I will use ListAllKeysFromIndex to get a list of keys for now. The
only reason that I included the m/r code is because of the error. If I get
another m/r job with similar output I need to know how to diagnose the
problem. I was using JavaScript m/r because I kind of understand JavaScript.
Is there a separate task to do Erlang m/r jobs.
Erlang phases can be added to a MapReduceQuery using MapErlang and
ReduceErlang.
I assume that I will need to know Erlang. Any recommendations on how best to
know what I need to know about Erlang to write a m/r job. But before I do
that wouldn't it be prudent to know that the source if the problem is indeed
JavaScript? How would I pinpoint that?
I think this has been answered on list. I'd search http://riak.markmail.org
Someone from Basho can probably handle this better than my handwaving that
using JavaScript involves an interpreter, type marshaling between Erlang and
JavaScript, and won't multi-thread like Erlang will.
These two m/r jobs are basically an example of using m/r that would be
typical for our application. Just for sheer maintenance we wouldn't want to
go down the path of maintaining a counter for all the fields that we have.
There could be departments, categories, celebrations, . . . Basically a lot
of them. For all intents it is an ad hoc query. If that is one of the
limitations then we will have to note it and see if coping with this
limitation is too onerous.
MR queries are going to scan all of your data on disk. If you have 5 nodes
that can read at ~100 MB/s and you have 100GB of data, how long will it take
for your ad hoc query to run?
Riak Search/Lucene/Yokozuna will be better options for ad hoc workloads than
MapReducing across the cluster.
From: riak-users [mailto:riak-users-boun...@lists.basho.com] On Behalf Of
Jeremiah Peschka
Sent: Tuesday, February 26, 2013 1:33 PM
To: riak-users
Subject: Re: MapReduce performance problem
Before you go troubleshooting performance problems, I'd focus on getting
results out of basho_bench and getting a good understanding of your
environment. If you're running 4 guests with 1 vCPU each on the same VM host
with all guests sharing a single pool of disks, no amount of tuning will
solve that problem. Without an understanding of the operating environment,
we can't do much more than point at general best practices and say "these
might help you, not sure, though."
As far as your specifics - for the first query, if you're attempting to get
a list of keys, I still recommend using ListAllKeysFromIndex(string Bucket).
This will be pushed out as part of the IRiakClient interface in the next day
or two, and I'm sure the gods of OOP won't kill you for using an actual
RiakClient object instead of an IRiakClient interface between now and then.
Sending those results back directly from riak_kv is going to be far faster
than messing around with a JavaScript MapReduce job.
Always keep in mind that MR jobs are not going to be the most efficient way
to perform any kind of ad hoc querying - they're great for large scale data
transformations but if you really want performance, you'll want to write
Erlang MR jobs.
If you need to maintain counts per department, a better approach will be
persisting counters and maintaining those counts via some kind of
caching/pre-aggregation mechanism, most likely outside of Riak because of
eventual consistency guarantees. Alex Siculars will eventually show up and
start chanting "use redis"; you'll be resistant at first, but his arguments
make a lot of sense. Riak does some things very well, maintaining consistent
counters isn't one of them... yet.
---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop
On Tue, Feb 26, 2013 at 10:52 AM, Kevin Burton <rkevinbur...@charter.net>
wrote:
I have a simple CorrugatedIron client that makes the following request:
IRiakClient riakClient = cluster.CreateClient();
RiakBinIndexRangeInput bucketKeyInput = new
RiakBinIndexRangeInput(productBucketName, "$key", "00000000", "99999999");
RiakMapReduceQuery query = new RiakMapReduceQuery()
.Inputs(bucketKeyInput)
.MapJs(m => m.Name("Riak.mapValuesJson").Keep(true));
RiakResult<RiakMapReduceResult> result =
riakClient.MapReduce(query);
So as you can see this is a very basic range m/r query. But the result comes
back as:
Riak returned an error. Code '0'. Message: timeout
CommunicationError
Another type of m/r query I have
IRiakClient riakClient = cluster.CreateClient();
var query = new RiakMapReduceQuery()
.Inputs(productBucketName)
.MapJs(m => m.Source(@"function(v,d,a) {" +
"var p = JSON.parse(v.values[0].data);" +
"var r = [];" +
"d = escape(p.Department);" +
"if(d != '') {" +
"var o = {};" +
"o[d] = 1;" +
"r.push(o);" +
"}" +
"return r;" +
"}"))
.ReduceJs(m => m.Source(@"function(v,d,a) {" +
"var r = {};" +
"for(var i in v) {" +
" for(var w in v[i]) {" +
" if(w in r) r[w] += v[i][w];" +
" else r[w] = v[i][w];" +
" }" +
"}" +
"return [r];" +
"}")
.Keep(true));
This returns but it takes far too long. I have about 60,000 items in my
bucket and this takes about 50-60 seconds to execute. The results seem
valid. For these types of m/r jobs what can I do on the server (or client)
to helo diagnose the problem. I have basic tools like iostat and top to
give me data but some pointers on using the output of these tools might
help.
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
