Re: Map Reduce Requirements

2011-08-23 Thread Brian Rowe
I'm a little late to the party, but the way I've been handling marshaling is using an explicit map/reduce phase to perform the marshaling and/or data massaging. You can chain map phases together by using the special bucket/key pair {none,none} and passing the intermediate data via the KeyData. This

Re: Map/Reduce In pure erlang

2011-07-29 Thread Brian Rowe
gt; > On Jul 29, 2011, at 9:09 AM, Brian Rowe wrote: > >> This isn't completely accurate. There's nothing to stop you from >> writing a module that reads in a key, compiles the payload as code, >> and calls a function within the newly compiled module. Similarly, you

Re: Map/Reduce In pure erlang

2011-07-29 Thread Brian Rowe
This isn't completely accurate. There's nothing to stop you from writing a module that reads in a key, compiles the payload as code, and calls a function within the newly compiled module. Similarly, you can set up the nodes to point to an external library path and run the modules from there. As fa

Updated Rekon and tips and tricks for Riak map/reduce jobs

2011-07-27 Thread Brian Rowe
gt; reduce, etc. I'm hoping I've got my facts right, but if I don't do let me know. http://cartesianfaith.wordpress.com/2011/07/27/mapreduce-tips-and-tricks-in-riak/ Regards, Brian Rowe ___ riak-users mailing list riak-users@lists.ba

Re: Riak compile error: Module gen_server2 potentially included by two different applications: rabbit_common and riak_core

2011-07-27 Thread Brian Rowe
Hello, I had this same problem with a different project (Nitrogen + Mochiweb). The problem is that in R14B03, there is an additional check within reltool to prevent duplicate modules from being packaged. IIRC the best of the kludges was to add an exclude directive in the reltool.config. HTH, Bria

Re: Reduce phase behavior clarification

2011-07-26 Thread Brian Rowe
b Ippolito wrote: >> >> It sounds like there is a typo, I would assume that it should be "the >> input of later evaluations will include the output of earlier >> evaluations". It's not really a reduction otherwise. >> >> On Mon, Jul 25, 2011 at 3:03

Reduce phase behavior clarification

2011-07-25 Thread Brian Rowe
Hello, I see the following on the Basho wiki for Riak [1]: The important thing to understand is that the function defining the reduce phase may be evaluated multiple times, and the input of later evaluations will include the input of earlier evaluations. For example, a reduce phase may implem

Introducing a forked Rekon that does map/reduce

2011-07-14 Thread Brian Rowe
Hello, I made some enhancements to Rekon that I thought the community might be interested in. The primary pay-off is the ability to execute map/reduce jobs over a bucket via Rekon. You can assemble phases together directly from the browser, execute the job, and see the results. The jobs can be any