Dear Matthew, The capacity adjustment at the Java script engines worked fine. I was able to run a clean-up on the database. I also decided o double the JS memory space to 16 MB, as RAM is not the issue on our server, There is still a doubt about the Map & Reduce syntax which is not giving a matching result:
My simple JavaScript, pattern scan & count works OK and delivers a counter
for a pattern search :
query.map("""function(v) { var str = '"""+search_string+"""'; var data =
JSON.parse(v.values[0].data); if
(v.values[0].data.toString().toUpperCase().search(str) != -1) { return
[[v.key, data]]; } return []; }""")
query.reduce("function(v) { var counteri = 0; for(var i in v) {counteri
+= 1;} return [['counter_found', counteri]]; } ")
The map function as you sent below returns a list of keys and 1 like this
query.map("""function(v) {if
(v.values[0].data.search('"""+search_string+"""') != -1) { return [[v.key,
1]]; } return []; }""")
How many keys in the bucket: 1000
agFa82470070 - 1
sDko48992336 - 1
eFxO66269553 - 1
Sovt26572593 - 1
..
This looks good. By using the JavaScript reduce function I get a zero
result
Source: query.reduce("function(v) { return
[v.reduce(function(acc,value){return acc + value;},0)]}")
Output: 0 - 0
Thanks in advance for clarification.
Regards,
Claude
Claude Falbriard
Certified IT Specialist L2 - Middleware
AMS Hortolândia / SP - Brazil
phone: +55 19 9837 0789
cell: +55 13 8117 3316
e-mail: clau...@br.ibm.com
blog: https://w3.tap.ibm.com/weblogs/inovarMF/
Project CallOwn Blue
From: Matthew Tovbin <matt...@tovbin.com>
To: clau...@br.ibm.com
Cc: riak-users@lists.basho.com
Date: 21/05/2012 17:01
Subject: Re: Riak Map / Reduce / Map capacity limit reached at
20000 keys
Sent by: tovb...@gmail.com
Claude,
1.
try to increase js_vm_counts as follows:
{riak_kv, [
...
{map_js_vm_count, 24 },
{reduce_js_vm_count, 18 },
...]
2. optimize your js function:
query.map("""function(v) {if
(v.values[0].data.search('"""+search_string+"""') != -1) { return [[v.key,
1]]; } return []; }""")
query.reduce("function(v) { return [v.reduce(function(acc,value){return
acc + value;},0]; } ")
-Matthew
On Mon, May 21, 2012 at 10:02 AM, <clau...@br.ibm.com> wrote:
Dear colleagues,
During my pilot testing I've got an error message during a text pattern
search operation coded as a Riak Client "map" logic. This error appeared
at a volume of approximately 20.000 keys, each with an 1 KB free text data
block associated. The Map function was coded in JavaScript. Here the
source:
query.map("""function(v) { var str = '"""+search_string+"""'; var data =
JSON.parse(v.values[0].data); if (v.values[0].data.toString().search(str)
!= -1) { return [[v.key, data]]; } return []; }""")
query.reduce("function(v) { var counteri = 0; for(var i in v) {counteri
+= 1;} return [['counter_found', counteri]]; } ")
The error shown is the following : [preflist_exhausted]
zbra:/opt/python/myprojects/riak # python pilot3.py
starts at : 2012-05-21 07:35:49.070470
How many keys in the bucket: 19760
Riak client exception: Error running MapReduce operation. Status: 500 :
{"phase":0,"error":"[preflist_exhausted]","input":"{ok,{r_object,<<\"ramtest\">>,<<\"XDHt25887344\">>,[{r_content,{dict,6,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[[<<\"Links\">>]],[],[],[],[],[],[],[],[[<<\"content-type\">>,97,112,112,108,105,99,97,116,105,111,110,47,106,115,111,110],[<<\"X-Riak-VTag\">>,53,86,72,48,120,77,53,99,114,86,111,77,77,54,109,114,79,70,54,52,49,109]],[[<<\"index\">>]],[],[[<<\"X-Riak-Last-Modified\">>|{1337,595548,374416}]],[],[[<<\"X-Riak-Meta\">>]]}}},<<\"{\"is_valid\":
true, \"cEDO\":
\"GQZumfdWBCfX...\">>}],...},...}","type":"forward_preflist","stack":"[]"}
Any suggestions how to improve the pattern scan capability in the in the
RiakClient map function, or how enhance the JavaScript code for better
capacity?
Are there any servers setup tuning options to adjust?
Thanks in advance for your support.
Regards,
Claude
Claude Falbriard
Certified IT Specialist L2 - Middleware
AMS Hortolândia / SP - Brazil
phone: +55 19 9837 0789
cell: +55 13 8117 3316
e-mail: clau...@br.ibm.com
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
<<image/jpeg>>
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
