Re: Writing a reduce phase with reduce_phase_only_1 and additional parameters

2012-06-01 Thread Bryan Fink
On Wed, May 23, 2012 at 3:03 PM, Manuel Gomez wrote: > This is my function so far: > > reduce_slice(WList,{Page,PageSize}) -> >   lager:info("Page and PageSize ~p - ~p",[Page,PageSize]), >   lists:sublist(WList, Page, PageSize). > > > This "works". The problem is that because the reduce phase gets

Re: Follow-up: Riak / Map, Reduce - error [preflist_exhausted]

2012-06-12 Thread Bryan Fink
On Tue, Jun 5, 2012 at 9:51 AM, wrote: > Any idea how to adjust the processing capacity of the Riak JavaScript in the > Map / Reduce process to avoid the error [preflist_exhausted] ? > Checking the user posts, I also see a similar error report sent at 29/05 by > Mr. Sati, Hohit. Hello, Claude an

Re: Link walking with Erlang riak client

2012-06-20 Thread Bryan Fink
On Tue, Jun 19, 2012 at 12:35 PM, Morgan Segalis wrote: > Hi everyone ! Hi, Morgan. > 1 - Are Link walking in Erlang fast & easy ? Link-walking in the Erlang client is provided via the MapReduce interface. Instead of a 'map' or 'reduce' phase, specify a 'link' phase like: {link, Bucket, Ta

Re: Link walking with Erlang riak client

2012-06-20 Thread Bryan Fink
On Wed, Jun 20, 2012 at 8:00 AM, Morgan Segalis wrote: > Why would you think that fetch both user and look through the links is > fastest than A -> B -> A ? Are links really that slow ? It's not that links are that slow, but that they're not that focused. Unless you're using B's key as the tag o

Re: link with out changing data

2012-06-26 Thread Bryan Fink
On Tue, Jun 26, 2012 at 2:43 AM, Venki Yedidha wrote: >     I want to insert Country data first, whenever a city > enters we should link this country to city with out changing the country > data. Hi, Venkatesh. As you have noticed, Riak does not support partial-update operations.

Re: link walking- get a single object from 1000 objects with tag 'city'

2012-06-26 Thread Bryan Fink
On Mon, Jun 25, 2012 at 6:06 AM, Venki Yedidha wrote: >   I have a document in which my Riak object is linked with > tag name 'city' to nearly 1000 other objects. >   However, If I use /bucketname/keyname/_,city,1 >                        It lists me all cities link

Re: About second parameter of riakc_pb_socket:mapred

2012-07-13 Thread Bryan Fink
On Thu, Jul 12, 2012 at 11:10 PM, 郎咸武 wrote: > But myself function is can not work(Inputs = {modfun, trend_riak, test, > [<<"a">>, <<"b">>]}). The "trend_riak" module is added riak path. > There are errors in the following: > {error,<<"Error sending inputs: [{<<\"my\">>,<<\"bu\">>}]">>} > > The tr

Re: Request queue issue with riakc_pb_socket

2012-08-02 Thread Bryan Fink
On Tue, Jul 24, 2012 at 8:48 PM, Julian wrote: > I'm using riakc_pb_socket to queue up several requests and rely on the > built-in queueing, which should hopefully eventually service all requests. > However I have a scenario in which a couple processes that queue a request > are never terminating

Re: MR Timeout

2012-08-03 Thread Bryan Fink
On Fri, Aug 3, 2012 at 10:50 AM, Yousuf Fauzan wrote: > The query fails with the following error > <<"{\"phase\":1,\"error\":\"{{{badmatch,[]},[{riak_kv_js_manager,needs_reload,2},{riak_kv_js_manager,handle_call,3},... Hi, Yousuf. I think there may be a race between a Javascript VM marking itself

Re: multi-get (yet again)

2012-08-10 Thread Bryan Fink
On Thu, Aug 9, 2012 at 5:11 AM, Kresten Krab Thorup wrote: > The only issue with this approach is AFAIK that M/R effectively runs with > R=1, i.e. it doesn't ensure that a value is consistent across replicas. > > IMHO riak_kv_mapreduce should have a map_get_object_value, which does a > proper Ri

Re: mapreduce with non-existent keys

2012-08-23 Thread Bryan Fink
Wow, this question slipped by while I wasn't looking. Sorry about that. On Mon, Jul 16, 2012 at 4:47 PM, Mark Boyd ソフトウェア 建築家 wrote: > Can anyone familiar with the innards of riak describe how distribution of a > map/reduce is handled when there are multiple reduce phases included as in > this so

Re: Bad MapReduce job brings the Riak to a screeching halt?

2012-08-30 Thread Bryan Fink
On Wed, Aug 29, 2012 at 11:07 PM, Brad Heller wrote: > The issue I'm facing: I tried to run an improperly-formatted MapReduce job > against a bucket with about 45k keys in it and it seemed to crash Riak. …snip… > So my question is: Why did this completely kill Riak? This makes me pretty > nervou

Re: 'forward_preflist' error in MapReduce

2012-08-31 Thread Bryan Fink
On Thu, Aug 30, 2012 at 7:58 PM, Matthew Tovbin wrote: > Recently I discovered that my MapReduce job, that used to run > successfully, stopped working. > Here is the error: > {"phase":0,"error":"[timeout]","input":"{<<\"test\">>,<<\"123\">>}","type":"forward_preflist","stack":"[]"} > > Is anyone f

Re: List of all available JS functions

2012-09-06 Thread Bryan Fink
On Thu, Sep 6, 2012 at 5:49 AM, shaharke wrote: > I've noticed that Riak extends the basic javascript functionality to add > syntactic-sugar functions on collections (e.g. forEach, filter, map etc.). > Is it possible the Riak simply uses the underscore.js library? If not, where > can I find a refe

Re: MapReduce job not stopping

2012-09-06 Thread Bryan Fink
On Thu, Sep 6, 2012 at 11:22 AM, shaharke wrote: > IMO it makes more sense that Riak will cancel the job once the connection > with the client is terminated. Is this a bug in Riak? Riak won't notice that an HTTP client has terminated the connection until it attempts to send some portion of the re

Re: riak_core application design questions

2012-10-02 Thread Bryan Fink
On Wed, Sep 26, 2012 at 4:38 PM, Chris Hicks wrote: > However, sending > the command off to, say, 3 nodes to process means that each of my nodes will > then ping Riak (if I need 2 responses per query suddenly that means I hit > Riak nodes a total of 6 times), process, and then all try to save the

Re: Map Reduce and long queries -

2012-10-15 Thread Bryan Fink
On Mon, Oct 15, 2012 at 4:13 AM, Olav Frengstad wrote: > Just as a note, using the Erlang pb client you can use the key filters > for 2i queries if you include the riak_kv_mapred_filters module in > your client code path. … > 2> Index = {index, <<"test">>, <<"$key">>, <<0>>, <<255>>}, > 2> {ok, Fi

Re: Is it possible to include node load into the consistency hashing function in riak-core?

2012-10-15 Thread Bryan Fink
On Tue, Oct 9, 2012 at 11:59 AM, Zhemzhitsky Sergey wrote: > What I’d like to achieve is to have the preferred list be sorted based on > the current load (maybe erlang:statistics(run_queue)) of the nodes. > > Is it possible to change the hashing function somehow so that the vnode on > the node wit

Re: noproc error on Map/Reduce

2012-10-15 Thread Bryan Fink
On Thu, Oct 11, 2012 at 12:12 AM, Matt Black wrote: > When I run the query it fails with this error, with which I am unfamiliar.. > Any thoughts on how to diagnose? Hi, Matt. The interesting part of that error is: > '{"phase":0,"error":"{noproc,{gen_server,call,[riak_kv_js_map,{reserve_vm,<0.234

Re: Riak mapreduce function map_object_value return empty result on erlang pb client

2012-10-15 Thread Bryan Fink
On Thu, Oct 11, 2012 at 1:32 AM, Mikhail Kuznetsov wrote: > I use riak erlang client for my project, and update riak and client library > today from master(client to 1.3.1 and server to 1.2.0). Everything works > except fetching multiple entities with map-reduce return empty. Instead of > usual re

Re: Riak mapreduce function map_object_value return empty result on erlang pb client

2012-10-15 Thread Bryan Fink
On Mon, Oct 15, 2012 at 12:34 PM, Mikhail Kuznetsov wrote: > We fixed it, but I am very angry… I agree. Breaking APIs without announcement is not okay. In this case, I don't think it was intentional. Instead, I think we just didn't know we broke this until your message arrived. In the latest vers

Re: read-repair and map-reduce

2012-11-05 Thread Bryan Fink
On Sun, Nov 4, 2012 at 2:03 AM, Igor Karymov wrote: > Hi all. Does I understand correctly that reading with mapreduce not trigger > read-repair mechanism? On Sun, Nov 4, 2012 at 9:45 AM, Igor Karymov wrote: > And how about link walking? It's same situation because link walking it's > just specia

Re: Rebar: user defined macro

2012-11-05 Thread Bryan Fink
On Fri, Nov 2, 2012 at 5:17 PM, Zabrane Mickael wrote: > I've this in my makefile: > > compile: > @$(REBAR) compile -Dfoo=17 ...snip... > How can I defined macro with value using rebar? Hi, Zabrane. It looks to me like the command line -D option for rebar does not support defining values

Re: Using map 'arg' in mapreduce calls in python and Erlang clients

2012-11-10 Thread Bryan Fink
On Fri, Nov 9, 2012 at 2:48 PM, Julian wrote: > map_reindex(RiakObject, _KeyData, IndexJson) when is_binary(IndexJson) -> > > and the error that the python client gets starts with Exception: > {"phase":0,"error":"function_clause" > > so I don't think it's being received as a binary. Hi, Julian. Y

Re: Pipe worker startup failed:fitting was gone before startup

2012-11-19 Thread Bryan Fink
On Mon, Nov 12, 2012 at 12:04 PM, Cornelius Schmale wrote: > I have some problems using riak and mapreduce queries. I’ve described the > hole problem at > > http://stackoverflow.com/questions/13345448/riak-fails-at-mapreduce-queries-which-configuration-to-use Hi, Cornelius. Could you describe a b

Re: MapReduce queries fail while node is starting

2012-11-19 Thread Bryan Fink
On Wed, Nov 14, 2012 at 12:04 PM, Nico Meyer wrote: > The following errors appear in the log of the node that handles the > MapReduce query: > > 2012-11-14 17:49:57.660 [error] emulator Error in process <0.32658.1777> on > node 'riak@172.20.1.11' with exit value: > {{badmatch,{'EXIT',timeout}},[{r

Re: Pipe worker startup failed:fitting was gone before startup

2012-11-21 Thread Bryan Fink
On Wed, Nov 21, 2012 at 2:10 PM, Cornelius Schmale wrote: > thanks for your excellent answer, it helped a lot for a deeper knowledge of > riak. > > You are right, I am using the default-one-node development cluster. After > decreasing the ring size to 8 as mentioned by you, everything works fine

Re: A bunch of Erlang client questions

2012-11-27 Thread Bryan Fink
Hi, Alex. I'm sorry you weren't able to find your answers in our docs. Some of them assume that you've read all of the other docs as well. I'll see what I can do to clear things up inline. On Mon, Nov 26, 2012 at 7:23 PM, Alex Rønne Petersen wrote: > * Is the proper way to replace the object asso

Re: A bunch of Erlang client questions

2012-11-27 Thread Bryan Fink
On Tue, Nov 27, 2012 at 11:41 AM, Alex Rønne Petersen wrote: >> In short, the proper way to replace an object is to first fetch the >> existing object, then update the value of what you fetched, then use >> riakc_pb_socket:put to write the new value. The reason for doing this >> is to comply with

Re: Using Secondary Indices as input to a map reduce function in Riak

2012-12-06 Thread Bryan Fink
On Thu, Dec 6, 2012 at 12:22 PM, Barry Ryan wrote: > I’m trying to use the output of a secondary index as the input to my > mapreduce function which is written in javascript using the HTTP API to > interact with Riak. I found on the Riak web-site that it is possible: 'If > you've enabled Seconda

Re: python map reduce and secondary indexes

2012-12-10 Thread Bryan Fink
On Sat, Dec 8, 2012 at 2:12 AM, David Montgomery wrote: > Given that map reduce is the primary way of getting data out of riak Hi, David. MapReduce is not the primary way of getting data out of Riak. Primary-key reads serve that purpose. Even for reading indexes, there are built-in interfaces (/b

Re: problem counting items with MapReduce

2012-12-10 Thread Bryan Fink
On Sat, Dec 8, 2012 at 6:11 AM, Antonio Rohman Fernandez < roh...@mahalostudio.com> wrote: > ** > > QUERY: > > { > "inputs":{"bucket":"sales","index":"sex_bin","key":"male"}, > "query":[ > {"map": {"language":"javascript","source":"function(v,k,a) { var m = > v.values[0].data.match(\"1981\

Re: error with MR and someone had painted it blue

2012-12-10 Thread Bryan Fink
On Sun, Dec 9, 2012 at 2:25 AM, David Montgomery wrote: > I am trying to run a mr. I get the below error. Other than a 500 error...I > dont see any other means to debug. What does the error mean? Hi, David. The clues are here: > '{"phase":0,"error":"[timeout]","input":"{<<\\"impressions\\">>,

Re: MapReduce job crashes?

2012-12-11 Thread Bryan Fink
On Tue, Dec 11, 2012 at 1:54 PM, Brad Heller wrote: > {error,[{phase,0},{error,<<"[{<<\"lineno\">>,459},{<<\"message\">>,<<\"InternalError: > script stack space quota is > exhausted\">>},{<<\"source\">>,<<\"unknown\">>}]">>} Hi, Brad. Your Javascript VM ran out of stack space. Raise the value o

Re: where did my data go when I write to riak?

2012-12-19 Thread Bryan Fink
On Wed, Dec 19, 2012 at 2:50 AM, David Montgomery wrote: > I am writing to riak and i get no errors. Then againgiven a key...,I > also get no data. Where did it go? How does one debug riak when you write > to riak..it looks like a success, get no data when get by key and get no > errors. H

Re: Getting Error reading /home/u1c332/riak-1.2.1/dev/dev1/etc/app.config

2012-12-20 Thread Bryan Fink
On Wed, Dec 19, 2012 at 6:01 PM, Evan Vigil-McClanahan wrote: > For some reason your version of erlang is printing the string "Running > Erlang" every time it starts. It doesn't normally do this. > > I am not sure where this error is coming from, as it doesn't seem to > be a normal part of start

Re: CRASH REPORT Process <<"TypeError: reduce of empty array with no initial value">>},{<<"source">>,<<"unknown">>}]}

2012-12-20 Thread Bryan Fink
On Thu, Dec 20, 2012 at 8:52 AM, David Montgomery wrote: > {fitting_exited_abnormally,[{<<"lineno">>,3},{<<"message">>,<<"TypeError: > reduce of empty array with no initial value">>} Hi, David. The error is occurring on line three of your reduce function, where it calls `values.reduce`: > query.

Re: Using the Local Client from a riak-attach session

2013-01-21 Thread Bryan Fink
On Fri, Jan 18, 2013 at 11:48 AM, Evan Vigil-McClanahan wrote: > Note that this uses the luke mapreduce subsystem, which isn't a good > idea as it's been deprecated going forward. > > Unfortunately there doesn't seem to be a way that's as simple to do > pipe mapreduce jobs. I'll ping people and s

Re: Differences between riak_client and riak_kv_mrc_pipe MapReduce when one node is down.

2013-01-31 Thread Bryan Fink
On Thu, Jan 31, 2013 at 6:07 AM, wrote: > Sorry John. You don't understand my question. > 1. One node(I mean physical(erlang) node) in cluster is down. > 2. It was down when i'm start job,when perform job and after it. We power off > this node. It's under repair. But we don't remove this node fr

Re: Differences between riak_client and riak_kv_mrc_pipe MapReduce when one node is down.

2013-01-31 Thread Bryan Fink
On Thu, Jan 31, 2013 at 1:46 PM, wrote: > I'm try fix this bug. > 1. Add fucntion for generating ConsHashCookie in riak_kv_mrc_pipe > ... > It's taking account of which nodes are up. > 2. Remove Hash = chash:key_of(ConstHashCookie) in reduce2pipe function. > > After that all work fine. Excellen

Re: Structure of Erlang MR Arg

2013-02-26 Thread Bryan Fink
Doh … sorry this thread ended up lost in the stack. Response below… On Feb 14, 2013, at 10:58 AM, Jeremiah Peschka wrote: > In the MapReduce Implementation docs [1], it looks like the Arg parameter of > an Erlang MR phase is not a single argument but is, in fact, a proplist of > Erlang terms,

Re: Structure of Erlang MR Arg

2013-02-26 Thread Bryan Fink
On Feb 26, 2013, at 9:16 AM, Bryan Fink wrote: >> {"reduce":{"language":"erlang", ... >> "arg":{"do_prereduce":true, >> "reduce_phase_batch_size": 250 ... I should have also mentioned that `d

Re: riak errors related to riak_pipe_vnode

2013-05-21 Thread Bryan Fink
Hello. A small update to this issue: While Alex is correct, that "fitting was gone before startup" messages in Riak logs can be a symptom of an overloaded cluster, they are most often nothing more than spam right after an MR query reaches its timeout. That is why I've just merged a change for the

Re: MapReduce: map then inputs

2013-05-24 Thread Bryan Fink
On Thu, Apr 4, 2013 at 3:59 PM, Jimmy Ho wrote: > inputs: [["user", "mathew"]], > query: [ {map:... get a list of [[bucket, friend_key]] } > > What is the next phase to read the bucket/key values as the new inputs? Hi, Jimmy. If that map phase is producing a list of [bucket, key] pairs, as you de

Re: Help with a complex Map Reduce query

2013-06-04 Thread Bryan Fink
Hi, Julien. On Sat, Jun 1, 2013 at 5:27 PM, Julien Genestoux wrote: > Yet, due to a bug in our implementation, we have 'lost' some entries. In > other words, some feedKey-entryKey elements are not in any feed object. … > Our initial solution was to list all the feed keys, and then, for each, Is

Re: Riak 1.3.1 Errors

2013-08-07 Thread Bryan Fink
On Wed, Aug 7, 2013 at 4:37 AM, Shane McEwan wrote: > It doesn't seem to be causing us any problems and @beerriot's comments on > #49 seem to indicate the messages are safe to ignore so that's what I'll do > until we're ready to upgrade. Indeed, the problem is harmless, other than generating log

Luwak External - Was: How to call riak_core_ring:remove_member/3 from erlang shell?

2014-08-20 Thread Bryan Fink
On Wed Aug 20, 2014, at 15:29:56 PDT, Sebastian Wittenkamp wrote: > we are running 1.0.0 since we are still using luwak. I don't think I've mentioned this on this list before, because it was mostly an experiment, and has never seen production use anywhere, but on the off chance it helps you dig ou

Re: worker_limit reached for map reduce jobs

2015-08-28 Thread Bryan Fink
On Thu, 27 Aug 2015 at 18:32:31 + Girish Shankarraman < gshankarra...@vmware.com> wrote: > > Hello, > > Currently using Riak 2.1.1. I have 7 nodes in my cluster. > I am looking for some information on how these worker_limits are > configured . > > > I have 50 linux(client) hosts trying to run

Re: Python for Luwak

2011-02-14 Thread Bryan Fink
On Mon, Feb 14, 2011 at 6:29 AM, Aditya Patadia wrote: > I have gone through documentation but could not find any python bindings for > luwak. Can anyone please guide me how can i store large files in Riak using > luwak and python??? Hello, Aditya. Unfortunately, there are no Python bindings spe

Re: luwak questions

2011-02-23 Thread Bryan Fink
On Tue, Feb 22, 2011 at 12:17 PM, francisco treacy wrote: >> db.add({ bucket: 'luwak_tld', key_filters: [['starts_with', >> 'e276814e96e0616eb7c07d3bb744d333']]}).map(function(v) { return [1] }).run() > POST /mapred >> { message: 'HTTP error 500: {"error":"bad_json"}' > , stack: [Getter/Setter] >

Re: Number of replica for Luwak

2011-02-23 Thread Bryan Fink
On Fri, Feb 18, 2011 at 8:54 PM, Les Mikesell wrote: > What happens if there is a read of the object while it is in the process of > being updated if the update is several different operations? Luwak streams work in an "all or nothing" fashion. That is, no read will see the result of any stream

Re: Number of replica for Luwak

2011-02-23 Thread Bryan Fink
On Wed, Feb 23, 2011 at 10:40 AM, Les Mikesell wrote: > On 2/23/2011 9:11 AM, Bryan Fink wrote: >> >> On Fri, Feb 18, 2011 at 8:54 PM, Les Mikesell >>  wrote: >>> >>> What happens if there is a read of the object while it is in the process >>&g

Moving mapred* functions out of riak_client?

2011-06-10 Thread Bryan Fink
ng like this: {ok, Results} = rpc:call(RiakNode, riak_kv_mapred_iface, mapred, [Inputs, Query]). Would this change be distasteful or unbearable for any of you? Thanks, Bryan Fink Senior Software Engineer Basho Technologies ___ riak-users mailing l

Announcing Riak Pipe (BETA)

2011-06-13 Thread Bryan Fink
ll in the early phases of creating these, though, so if exploring the riak_pipe repository gives you ideas, please don't hesitate to get in touch. And, again, Riak Pipe is BETA software. Basho does not support running it in production at this time. Cheers, Bryan Fin

Re: Moving mapred* functions out of riak_client?

2011-06-13 Thread Bryan Fink
On Fri, Jun 10, 2011 at 11:18 AM, Kresten Krab Thorup wrote: > We are using those APIs in our new stuff for riak sync/mobile. My problem > with your change would be that we have a mock riak_client which is super > convenient for unit testing, and it would be good to still be able to have > that

Re: Moving mapred* functions out of riak_client?

2011-06-13 Thread Bryan Fink
On Mon, Jun 13, 2011 at 4:01 PM, Les Mikesell wrote: > On 6/13/2011 2:40 PM, Bryan Fink wrote: >> >> Or if I could take a look at your mock system, maybe I could help with >> some suggestions for moving in the direction I proposed?  We are >> generally trying to move to

Re: mr_queue gone wild

2011-07-05 Thread Bryan Fink
because a single logical map phase gets split into a separate map request for each vnode. So, you may have only one MapReduce request outstanding, but see numbers greater than 1 in this output. Hope this helps, Bryan Fink Senior Software Developer, Basho Technologies

Re: luwak backend and misc.

2011-08-08 Thread Bryan Fink
On Thu, Jul 28, 2011 at 12:00 PM, Kunal Nawale wrote: > Hi, >  I am evaluating luwak to be used as a redundant file storage server. I am > trying to find out which backend will better suit for my purpose. Each of my > server has sixteen 1TB drives, 4 total servers, 48GB ram each, 1x10Gb > network

Re: odd error on mapreduce

2011-08-26 Thread Bryan Fink
On Fri, Aug 26, 2011 at 6:32 AM, Giancarlo Frison wrote: > If i serialize the body as Json I catch errors any more instead of protobuf > object representation (application/octet-stream). > Would be an mapreduce bug? > Is it possibile to skirt any body parsing during the mapreduce? Just for > evalu

Re: Riak 1.0 pre2 legacy_keylisting crash

2011-09-12 Thread Bryan Fink
Hi, Ryan. From the log messages you've pasted so far, it looks like something decided to shutdown the MapReduce pipe explicitly ("Reason: killed"). That can happen if there is a problem sending inputs, or if the request times out. In either of those cases, you should have received an error on th

Re: MR Output Changes

2011-09-26 Thread Bryan Fink
On Sat, Sep 24, 2011 at 4:39 PM, Jeremiah Peschka wrote: > MapReduce phases with a keep set to true are returning in multiple > RpbMapRedResp messages. I expected this, especially since hte API says it's > possible. What's causing the issue is the output format. … > Is there a reason that Riak i

Re: luwak_file:create does not truncate file content?

2011-09-26 Thread Bryan Fink
On Mon, Sep 26, 2011 at 1:04 AM, vuleetu wrote: > I tried many times and found seems like as long as i did these operation > using the same Riak connection, it will show results above. but when i used > another Riak connection({ok, Riak} = riak:client_connect('riak@127.0.0.1'). > ), it will trunca

Re: 500 Internal Server Error when performing link walking

2011-09-27 Thread Bryan Fink
On Tue, Sep 27, 2011 at 10:51 AM, Ivaylo Panitchkov wrote: > http://192.168.1.69:8098/riak/hb/first/_,_,1/_,_,0 Hi, Ivaylo. What Riak is trying to tell you (though it's being rather obtuse about it) is that the query you sent doesn't make sense to it. The problem is that '0' on the final link st

Re: Building a native Riak executable

2011-10-04 Thread Bryan Fink
On Tue, Oct 4, 2011 at 4:29 AM, Olivier Rossel wrote: > I need to deploy Riak on a Linux server with no Erlang VM. > Is there a procedure to build a native Riak executable on my own Ubuntu > so I can deploy it on this server? > Which Erlang .deb packages are required on my Ubuntu to build this nat

Re: application_start_failure, wriaki, {{{badmatch, {error, econnrefused}

2011-10-06 Thread Bryan Fink
On Thu, Oct 6, 2011 at 2:56 AM, Norman Khine wrote: > my riak cluster is up and running: > > http://pastie.org/2648028 … > aqoon:wriaki khinester$ vim rel/wriaki/etc/app.config > > modified the web_port to  {web_port, 8098} … > =INFO REPORT 6-Oct-2011::08:37:47 === >    application: wriaki >  

Re: Riak 1.0 pre2 legacy_keylisting crash

2011-10-07 Thread Bryan Fink
On Fri, Oct 7, 2011 at 1:50 AM, Fyodor Yarochkin wrote: > Here's one of the queries that consistently generates series of > 'fitting_died' log messages: > > { >   "inputs":{ >       "bucket":"test", >       "index":"integer_int", … >   }, >   "query":[ >    {"map":{"language":"javascript", … >    

Re: Using CommonJS module in Riak mapreduce/pre-commit hook

2011-10-24 Thread Bryan Fink
On Mon, Oct 24, 2011 at 12:08 PM, briche arnaud wrote: > Hi, >    I just want to know if one can import CommonJS modulesin Riak mapreduce > or pre-commit hook function. Hi, Briche. I'm unfamiliar with how the CommonJS modules are structured, but if they're available as a simple collections of .j

Re: problem with baseball MR example?

2011-10-24 Thread Bryan Fink
On Fri, Oct 7, 2011 at 8:56 PM, Nate Lawson wrote: > Perhaps this is the flaw the author hints at? If so, what's the proper way in > Riak MR to specify a split function to be sure proper boundaries are applied > to Luwak files? Hi, Nate. Indeed, you found the flaw. If you haven't stumbled on

Re: Newline in Key Legal

2011-10-25 Thread Bryan Fink
On Tue, Oct 25, 2011 at 12:54 AM, Jim Adler wrote: > Ok, this is a little simpler but I'm not sure how to proceed. When I pull > the keys via ?keys=stream API, I have a key "name-this\nBob". But > fetching it gets me a 404 not found error.  I've tried %0D%0A (and other > variants) in the API call

Re: Newline in Key Legal

2011-10-25 Thread Bryan Fink
On Tue, Oct 25, 2011 at 10:35 AM, Jim Adler wrote: > Hi Bryan - I'm now running 1.0.1 but the keys were inserted under 0.14.  I > tried your test and it worked fine.  So, maybe I'm caught between > inserting under an old version and now reading under another? It's possible that your upgraded node

Re: Strange MapReduce behavior

2011-10-26 Thread Bryan Fink
On Wed, Oct 26, 2011 at 2:53 PM, Elias Levy wrote: > For instance: > > mr = Riak::MapReduce.new(riak) > mr.add('bucket', 'key') > mr.reduce("function(v) { return [] }", :keep => true)) > This returns: > {"bucket"=>"key"} I believe this one is explainable by a known Riak 1.0 MapReduce bug: https:

Re: Tuning parameters for reduce phases introduced in 1.0

2011-10-27 Thread Bryan Fink
On Wed, Oct 26, 2011 at 6:48 PM, Elias Levy wrote: > Riak 1.0 introduced two reduce phase tuning > parameters: reduce_phase_batch_size and reduce_phase_only_1. Both of these > can be specified on a per phase basis using the arg parameter, passing them > in as a map.  This appears to overload the a

Distributed reduce phases via pre-reduce [Was "Secondary Indexes - Feedback?"]

2011-11-18 Thread Bryan Fink
On Thu, Nov 17, 2011 at 9:45 AM, Gordon Tillman wrote: > I'm really interested in being able to implement distributed > reduce phases (specifically to do a partial sort)  and then have that output > handle by a final reduce phase that could perform an efficient merge sort >  and stream results bac

Re: Setting up a project on top of Riak

2011-11-28 Thread Bryan Fink
On Sat, Nov 26, 2011 at 10:56 AM, Mathieu D'Amours wrote: > How should we set up a development environment so that Riak and all other > Erlang applications works, and lets us build our different stuff (storage > backend, vnodes, web app) on top of it? What's a good way of having good > versatil

Re: "consistent" map/reduce

2011-11-28 Thread Bryan Fink
On Mon, Nov 21, 2011 at 2:46 PM, Kresten Krab Thorup wrote: > I'd like to be able to do a "consistent map/reduce" job i.e., with "R=2 > semantics" for an "N=3 bucket".  Maybe other people have the same need, but I > can't see if this is possible ... perhaps with the new riak_pipe > infrastructu

Re: Luwak PUT Content-Range

2011-11-29 Thread Bryan Fink
On Tue, Nov 29, 2011 at 6:07 AM, John Axel Eriksson wrote: > Is it possible to incrementally add to a file in Luwak using PUT and the > Content-Range header. Hi, John. At this time, it is not possible to use Content-Range when PUTting Luwak content. > The reason I want to do this is because we

Re: MapReduce Weird Issue

2011-11-29 Thread Bryan Fink
On Tue, Nov 29, 2011 at 3:56 AM, Yehuda Zargrov wrote:>      .reduce("function(v) {var s={}; for(var i in v)  { var date=''; var> sum=0; for(var n in v[i]) { if (n === \"date\") date=v[i][n]; if (n ===> \"sum\") sum=v[i][n]; } if (date in s) s[date] += sum; else s[date] = sum; }> return[v]; }", :k

Re: Configurable R value for MR jobs?

2011-11-30 Thread Bryan Fink
On Tue, Nov 29, 2011 at 8:01 PM, Elias Levy wrote: > Is there such thing as a configurable R value for MR jobs? No, there is not. Effectively, R=1 for all MR jobs currently. You may be interested in following a conversation that Kresten Krab Thorup and I are having on this issue: http://lists.b

Re: "consistent" map/reduce

2011-11-30 Thread Bryan Fink
On Wed, Nov 30, 2011 at 8:36 AM, Kresten Krab Thorup wrote:> 1. How do I create the initial inputs? i.e. the list of all {Index, Node} pairs that go into the riak_kv_pipe_listkeys fitting.  Does this fitting need a special chashfun to send it to the right vnode? The easiest way to get the "go" mes

Re: "consistent" map/reduce

2011-11-30 Thread Bryan Fink
Ew. Major apologies for the formatting in my last response. I've no idea what happened. Let me know if it's unreadable, and I'll try sending again. -Bryan ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/

Re: "consistent" map/reduce

2011-12-01 Thread Bryan Fink
On Wed, Nov 30, 2011 at 4:56 PM, Kresten Krab Thorup wrote:> Now, it seems that if the items passing between the map phase and reduce phase are many and small, then the system could benefit from a "chunking" fitting that collects items and sends them off in a list to the next fitting, after receiv

Re: Secondary Index Map and reduce order and performance

2011-12-01 Thread Bryan Fink
On Wed, Nov 30, 2011 at 4:28 PM, Sajithkumar Kizhakkiniyil wrote: > If my understanding is correct a reduce phase with > riak_kv_mapreduce.reduce_identity is needed for secondary index query. Hi, Sajithkumar. The reduce_identity function is only needed if the result you want is the list of bucke

Re: Timeout during reduce phase

2011-12-13 Thread Bryan Fink
On Mon, Dec 12, 2011 at 1:12 PM, Matthew A. Brown wrote: > The map/reduce itself has a timeout of 10 minutes, but this is > happening after about 30 seconds. Any idea what could be causing this? Hi, Mat. Is your reduce phase implemented in Javascript? If so, it's subject to a hardcoded five sec

Re: Weird (?) error while using riak erlang pb client's mapreduce

2012-01-06 Thread Bryan Fink
On Fri, Jan 6, 2012 at 10:18 AM, Alin Popa wrote: > and this is the code that I'm using for this: > > -module(simple_mapreduce). > > -compile(export_all). > > main(Pid) -> >   {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087), >   Fun = fun(Object, _KeyData, none) -> [Object] end, >   riak

Re: error: undef done()

2012-02-01 Thread Bryan Fink
On Tue, Jan 31, 2012 at 1:46 PM, Matthew A. Brown wrote: > [error] error:undef done() > > The error message is followed by a dump of the reduce inputs which is, > sadly, truncated before the bit where it would give the call trace. > > Our Erlang reduce code does not define or reference a function

Re: error: undef done()

2012-02-01 Thread Bryan Fink
On Wed, Feb 1, 2012 at 11:43 AM, Matthew A. Brown wrote: > Thanks much for the info. We're running Riak 1.0.3. I think in our > case what would've made this easier to track down would have been not > having the error output cut off in the log -- since the inputs to the > reduce comprised a pretty

Re: Performance of link walking versus map/reduce

2012-02-03 Thread Bryan Fink
On Fri, Feb 3, 2012 at 12:31 PM, Nicolas Petton wrote: > curl > http://localhost:8098/buckets/artists/keys/pink_floyd/albums,author,_ > was nearly immediate, it took 0.156s > > while: > > curl -X POST -H "content-type:application/json" \ >  http://localhost:8098/mapred --data @- > {"inputs":[["art

Re: "forward_preflist" timeout error after 1.1 upgrade

2012-02-23 Thread Bryan Fink
On Wed, Feb 22, 2012 at 11:26 AM, Matthew A. Brown wrote: > Hi all, > > We're seeing a new timeout error after upgrading our cluster to Riak > 1.1. The error message: > > {"phase":0,"error":"[timeout]","input":"{{<<\"service_profiles\">>,<<\"8fh/2\">>},{struct,[{<<\"key\">>,<<\"s-10925\">>}]}}","t

Re: recursive/looping map-reduce

2012-02-23 Thread Bryan Fink
On Wed, Feb 22, 2012 at 8:23 PM, Michael Radford wrote: > I'm trying to figure out whether there's an easy way to accomplish > something like a "recursive/looping map-reduce query" in Riak. > … > > Are there any other options short of running custom query-coordinating > code at the level of riak_p

Re: slow mapred_search key lookups for single terms

2012-04-05 Thread Bryan Fink
On Wed, Apr 4, 2012 at 11:28 AM, Michael Radford wrote: > Aha, I just noticed that the native erlang client is still using > luke_flow to implement its map-reduce, rather than riak_pipe.  On some > level, this must be the reason for the differing behavior...either a > bug in riak_pipe, or a bug in

Re: slow mapred_search key lookups for single terms

2012-04-06 Thread Bryan Fink
On Thu, Apr 5, 2012 at 3:51 PM, Bryan Fink wrote: > This *might* be the wrong intuition for Search, since there is > funneling happening to process the query anyway, but it's likely a > good place to start. Whoops. I *really* should have included a suggestion for how to figure ou

Re: slow mapred_search key lookups for single terms

2012-04-06 Thread Bryan Fink
On Fri, Apr 6, 2012 at 11:00 AM, Michael Radford wrote: > Calling riak_kv_mrc_pipe directly from the console is just as slow as > using the pb interface. Thank you for confirming that. Much appreciated. > it seemed to be encoding one PB message per search result and > spending quite a bit of ti

Re: list keys with key filters

2012-04-23 Thread Bryan Fink
On Sat, Apr 21, 2012 at 12:39 PM, Sam Lang wrote: > When I use ascii or json objects, I don't get the exception, and the keys get > listed properly.  I assume that riak is trying to parse the data of my binary > object and failing because the data isn't utf8 encoded.  Is it possible to do > thi

Re: list keys with key filters

2012-04-23 Thread Bryan Fink
On Mon, Apr 23, 2012 at 4:18 PM, Sam Lang wrote: > That worked, thanks Bryan!   Any way to avoid sending the values of the > object to the client as well? What I pasted shouldn't be sending the values of the objects to the client. It should send just the keys back. I'm not a major user of the

Re: Setting and getting post-commit hooks via the erlang http client

2012-04-28 Thread Bryan Fink
On Fri, Apr 20, 2012 at 11:03 AM, Antoine wrote: > It looks like it is not possible to set (neither get) a postcommit hook via > the riak erlang http client. > > The postcommit bucket property is set to be ignored in > rhc_bucket:httpify_prop/2. > Is there any reason for that? It looks like a sim

Big Changes in Riak Tip

2010-04-13 Thread Bryan Fink
Hello, Riak users. We wanted to give you all a heads up that we have just pulled a large changelog into the tip of the Riak repository. We have been working on this list of changes privately, in order to cause less confusion, as it covers a fairly major change, that was experimental at first. Th

Re: Big Changes in Riak Tip

2010-04-14 Thread Bryan Fink
On Tue, Apr 13, 2010 at 4:37 PM, Ali Sabil wrote: > > Are there any plans to split this further and have riak_core, riak_kv > and riak_http ? > > Ali > Hi, Ali. I have a couple of answers, depending on the intent of your question. The first answer, is that the code actually handling HTTP connec

[ANN] Riak Release 0.10

2010-04-23 Thread Bryan Fink
Hello, Riak users. We are excited to announce the release of Riak version 0.10. There's a great writeup about the new features on the Basho Blog: http://blog.basho.com/2010/04/23/riak-0.10-is-full-of-great-stuff/ Pre-built installations and source tarballs are available at: http://downloads.bash

[ANN] Riak version 0.10.1

2010-04-26 Thread Bryan Fink
Hello, again, Riak users. We fixed a couple of small issues over the weekend, and decided to tag and release them as 0.10.1. Downloads are here: http://downloads.basho.com/ Release notes here (and below): http://downloads.basho.com/riak/riak-0.10/riak-0.10.1.txt Cheers, The Basho Team ---

Re: Error When Creating an Erlang Riak Object

2010-05-07 Thread Bryan Fink
On Fri, May 7, 2010 at 8:43 AM, Ted Karmel wrote: > Thanks Abe! > > That solved the error.  The readme should perhaps be updated to reflect this Done. Thanks for reporting the trouble! -Bryan ___ riak-users mailing list riak-users@lists.basho.com htt

  1   2   >