basho bench - custom value generator

2010-09-13 Thread Joseph Lambert
Hi, I'm trying to create a custom value generator for Basho Bench, but keep getting this error: escript: exception error: no match of right hand side value {error, {bad_return, {{basho_bench_app,start,[normal,[]]},

Map reduce error on large bucket

2010-09-13 Thread SKester
I am trying to run a map-reduce job against all keys in a bucket. The job is working fine on buckets with ~60,000 or less entries. However on buckets with > 63,000 keys I get the following error every time: Input: curl -X POST -H "content-type: application/json" http://testdw0b01.be.weather.com

Re: Map reduce error on large bucket

2010-09-13 Thread Alexander Sicular
Hmm... sounds to me like your'e getting close to a magic comp sci number. 2^16 or 65536. If this isn't a timeout thing, which it could be [1], it could be a bug. -Alexander [1] https://issues.basho.com/show_bug.cgi?id=367 On Sep 13, 2010, at 2:05 PM, SKester wrote: > I am trying to run a map-

Re: Map reduce error on large bucket

2010-09-13 Thread Sean Cribbs
There have been many improvements to MapReduce on the "tip" version, largely thanks to some customers who want to crunch large datasets. In some test runs, we have processed over 400K keys, so we know it's possible. Expect these improvements in the 0.13 release (which will be pretty soon, if you

Re: Map reduce error on large bucket

2010-09-13 Thread Kevin Smith
One problem with using the default number of partitions, which is 64, is that many bucket/key pairs wind up hashing to the same few partitions. This has the effect of concentrating the majority of the map function invocations onto a few processes which winds up causing read function outs. These

Riak Recap for 9/10 - 9/12

2010-09-13 Thread Mark Phillips
Afternoon, Evening, Morning to all, Hope everyone had a good weekend. For today's Recap: blogs, Gists, events, a WebUI (with a challenge), and more. Enjoy! Mark Community Manager Basho Technologies wiki.basho.com twitter.com/pharkmilliups Riak Recap for 9/10 - 9/12 1) @TheColonial put togethe

Re: Native map-reducing with local interface?

2010-09-13 Thread Gustavo Niemeyer
> This is a feature we've definitely thought about. We have it on our roadmap > of improvements but I can't say when this feature will appear in Riak. We are > committed to making MapReduce more useful and usable for everyone and > allowing external processes to participate in MapReduce jobs is

Re: basho bench - custom value generator

2010-09-13 Thread Joseph Lambert
I found that changing erlang:apply(Module, Function, [Id] ++ Args); to Module:Function([Id] ++ Args); in basho_bench_valgen, got rid of the error. I'm not sure the reason why. - Joe Lambert joseph.g.lamb...@gmail.com On Mon, Sep 13, 2010 at 6:38 PM, Joseph Lambert wrote: > Hi, > > I'm try