Re: Luwak PUT Content-Range

Well, I don't know what I'm doing wrong here but I'm mainly using ruby. Excon (which Ripple uses) doesn't seem to have a way to pass an IO to a put request (am I missing something?). I've also tried NetHTTP in ruby which is actually supposed to be able to do that and I've tried Javas HttpURLConnec

Riak search performance FUD

Hi all, I'm currently evaluating the search functionality of Riak. This involves porting an application from Postgres/Sphinx to possibly only Riak. The application I'm porting doesn't need advanced search, but it does need a level of search I have come to believe this isn't provided in a feasible

Re: Configurable R value for MR jobs?

On Tue, Nov 29, 2011 at 8:01 PM, Elias Levy wrote: > Is there such thing as a configurable R value for MR jobs? No, there is not. Effectively, R=1 for all MR jobs currently. You may be interested in following a conversation that Kresten Krab Thorup and I are having on this issue: http://lists.b

Re: "consistent" map/reduce

Hi Bryan, now I've implemented the bulk of this, and obviously have some follow-up questions: 1. How do I create the initial inputs? i.e. the list of all {Index, Node} pairs that go into the riak_kv_pipe_listkeys fitting. Does this fitting need a special chashfun to send it to the right vnod

Re: Is Riak appropriate for website metrics?

I agree with the suggestion of gathering a bulk of analytics and then flushing to Riak, especially as each record is so small, which seems the overhead per record vs size of actual record seems excessive. I'd consider grouping your analytics into daily blobs, if not hourly. Writing directly to Riak

Incorect Input to Reduce

Hi, I've posted a mail on this few days ago, but went a bit further since then. I'm using ripple, running a map function on some input and getting a list of about 50 items. Right after that, I'm running a reduce function. The first thing I do in it is putting the length of v (the input) as an

Re: "consistent" map/reduce

On Wed, Nov 30, 2011 at 8:36 AM, Kresten Krab Thorup wrote:> 1. How do I create the initial inputs? i.e. the list of all {Index, Node} pairs that go into the riak_kv_pipe_listkeys fitting.  Does this fitting need a special chashfun to send it to the right vnode? The easiest way to get the "go" mes

Re: Incorect Input to Reduce

Hi Yehuda, Your reduce function is not guaranteed to run across all inputs in the first application, but may be applied many times as new inputs come in. Riak will batch up reduce inputs until a certain number or timeout is reached, then apply the function and take its output to be part of the inp

Re: Incorect Input to Reduce

The reduce phase receives data in batches. Per the docs, the default size is for every 20 inputs, reduce. The batch size can be tuned via config or per m/r request. Re: 13, likely what you're seeing is the length of the last phase sent to reduce. It seems if I return [values.length], I get just th

Re: "consistent" map/reduce

Ew. Major apologies for the formatting in my last response. I've no idea what happened. Let me know if it's unreadable, and I'll try sending again. -Bryan ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/

Re: Incorect Input to Reduce

Excellent tip Sean, works great for a similar issue I was having. If I already had args being passed to my reduce phase though (such as when calling Riak.reduceSlice), is it possible to specify only_1 and still ensure that my regular args pass through to the function as expected? For instance, redu

Re: Incorect Input to Reduce

n/m, I just need to modify my args from [0,15] to {"reduce_phase_only_1":true,0:10,1:15}* Jonathan Langevin Manager, Information Technology Loom Inc. Wilmington, NC: (910) 241-0433 - jlange...@loomlearning.com - www.loomlearning.com - Skype: intel352 * On Wed, No

Re: Incorect Input to Reduce

To be more clear for others reading, and correct my example: > [0,15] to {"reduce_phase_only_1":true,0:10,1:15} [0,15] to {"reduce_phase_only_1":true,"0":0,"1":15} (quoting the integer keys is not necessary, doing so for clarity) * Jonathan Langevin Manager, I

Re: Unexpected issue w/ m/r Riak.reduceSlice

Based on a quite helpful tip from Sean via another thread, here's my working solution :-) It was indeed an issue with how batches were being processed, so running the reduce phase as run-once, fixes the issue. { "inputs":"demo_3_Course", "query":[ { "map":{ "keep":

Re: riak-users Digest, Vol 28, Issue 55

On Wed, Nov 30, 2011 at 6:01 AM, wrote: > From: Jeroen van Dijk > > The use case I'm talking about is when you are looking for a term that is > very common and thus will yield many results. My understanding of the > implementation of Riak [citation needed] is that the search is divided into > a

Re: Riak search performance FUD

Hi Jeroen, Your understanding is correct, the search query is parsed into a tree, where each leaf of the tree corresponds to a term. Each leaf sends back all matching terms, and results are intersected (or unioned) where the branches come together. So yes, if you were to run a search on a term wit

Re: Secondary Indexes - Feedback?

Here at Clipboard, we make very heavy use of Riak Search and a couple of manual indices here and there. I've wanted to use 2i a few times but have decided against it for a few reasons: 1) Apprehension about the coverage set query, as Matt articulated. 2) Lack of ordering of returned results. Ge

Erlang Port Configuration

I'm setting up a new cluster with an iptables firewall, and I'm trying to figure out which ports to open. It looks like I need 4369, 8099, 8098 (http), and a configurable range. Can I configure the range to be a single port, or is there a minimum number of required ports? What determines how ma

Re: Erlang Port Configuration

On Wed, Nov 30, 2011 at 01:50:48PM -0600, Paul Gross wrote: > I'm setting up a new cluster with an iptables firewall, and I'm > trying to figure out which ports to open. It looks like I need 4369, > 8099, 8098 (http), and a configurable range. Can I configure the > range to be a single port, or is

Re: Erlang Port Configuration

What Andrew said… AFAIK, the only thing you can not configure is which ip address the erlang vm listens on. I think it will listen on all available ip addresses (0.0.0.0). Note that this is different from the riak process itself, which can be limited to a specific ip. That's at least what I came

Re: Erlang Port Configuration

I'm setting up a new cluster with an iptables firewall, and I'm trying to figure out which ports to open. It looks like I need 4369, 8099, 8098 (http), and a configurable range. Can I configure the range to be a single port, or is there a minimum number of required ports? What determines how many

Secondary Index Map and reduce order and performance

Hello Probably my understanding of M/R might be wrong. But I am getting drastic performance difference when running secondary index query on PB with map and reduce function in different order. If my understanding is correct a reduce phase with riak_kv_mapreduce.reduce_identity is needed for seco

Re: Secondary Index Map and reduce order and performance

Do you get the results in both cases? -Alexander Sicular @siculars http://siculars.posterous.com On Nov 30, 2011, at 4:28 PM, Sajithkumar Kizhakkiniyil wrote: > Hello > Probably my understanding of M/R might be wrong. But I am getting drastic > performance difference when running secondary ind

Re: Secondary Index Map and reduce order and performance

Do you get the *same* results in both cases? -Alexander Sicular @siculars http://siculars.posterous.com On Nov 30, 2011, at 4:28 PM, Sajithkumar Kizhakkiniyil wrote: > Hello > Probably my understanding of M/R might be wrong. But I am getting drastic > performance difference when running second

Re: Erlang Port Configuration

On Wed, Nov 30, 2011 at 02:09:45PM -0600, Paul Gross wrote: > >>I'm setting up a new cluster with an iptables firewall, and I'm > >>trying to figure out which ports to open. It looks like I need 4369, > >>8099, 8098 (http), and a configurable range. Can I configure the > >>range to be a single port

RE: Secondary Index Map and reduce order and performance

In my testing I did get the same result. My scenario is simple I created 200 keys with the same 2i key/value and retrieved it. Regards Sajith From: Alexander Sicular [mailto:sicul...@gmail.com] Sent: Wednesday, November 30, 2011 1:32 PM To: Sajithkumar Kizhakkiniyil Cc: riak-users@lists.basho.c

Re: "consistent" map/reduce

Thanks for your answers. Now, it seems that if the items passing between the map phase and reduce phase are many and small, then the system could benefit from a "chunking" fitting that collects items and sends them off in a list to the next fitting, after receiving X # of items, or after some t

Re: Secondary Indexes - Feedback?

On Wed, Nov 30, 2011 at 1:32 PM, wrote: > Here at Clipboard, we make very heavy use of Riak Search and a couple of > manual indices here and there. I've wanted to use 2i a few times but have > decided against it for a few reasons: > > 1) Apprehension about the coverage set query, as Matt articula

Riak Recap for November 28 - 30

Afternoon, Evening, Morning to All - Huge Recap for today: new code, presentations, blogs, and more. Enjoy. Mark Community Manager Basho Technologies wiki.basho.com twitter.com/pharkmillups Riak Recap for November 28 - 30 1) We rel

Re: Question: {error,{error,insufficient_vnodes_available}}

Hi Dan, Thanks for your reply. Indeed, I saw the "insufficient_vnodes_available" error message with only one node on the machine. What we did was just install the official 1.0.2's deb version and modifying vm.args and app.config. The only changes in our setting are: A. app.config: - (http ip .. -

Re: Riak Recap for November 28 - 30

I keep seeing CRDT inspired frameworks. Here's one I built in Python that someone may find helpful. It's a bit of a toy but it has good test coverage for the datatypes I've implemented. Framework: https://github.com/ericmoritz/crdt Riak Adapter: https://github.com/ericmoritz/riak_crdt On Wed,

Re: Erlang Port Configuration

A range of 1 might work, but I'd go for a range of 5 or so, just to be safe. Thanks for the guideline. Can you give me some insight on when riak would use more than one port? Thanks, Paul www.pgrs.net ___ riak-users mailing list riak-users@lists.bash

Re: Riak Recap for November 28 - 30

On Wed, Nov 30, 2011 at 5:23 PM, Mark Phillips wrote: [snip] > 2) Basho and Riak are going to have a significant presence in > Australia over the next few weeks. > > * OJ Reeves, one of the author's of Riak's .NET Client, will be giving > a talk at Erlang Factory Lite in Brisbin called "Riak: wh

Re: Erlang Port Configuration

On Wed, Nov 30, 2011 at 10:12:24PM -0600, Paul Gross wrote: > >A range of 1 might work, but I'd go for a range of 5 or so, just to be > >safe. > Thanks for the guideline. Can you give me some insight on when riak > would use more than one port? > The most likely scenario is if riak crashes, and by