Based on a quite helpful tip from Sean via another thread, here's my
working solution :-)
It was indeed an issue with how batches were being processed, so running
the reduce phase as run-once, fixes the issue.
{
"inputs":"demo_3_Course",
"query":[
{
"map":{
"keep":true,
"language":"javascript",
"arg":null,
"source":"function(value){return [value.key];}"
}
},
{
"reduce":{
"keep":true,
"language":"javascript",
"arg":{"sort":"function(a,b){
if (a < b) {
return -1;
} else if (a === b) {
return 0;
} else if (a > b) {
return 1;
}
}","slice":[80,85],"reduce_phase_only_1":true},
"source":"function(values,arg){return
Riak.reduceSlice(Riak.reduceSort(values,arg[\"sort\"]),arg[\"slice\"]);}"
}
}
]
}
*
<http://www.loomlearning.com/>
Jonathan Langevin
Manager, Information Technology
Loom Inc.
Wilmington, NC: (910) 241-0433 - jlange...@loomlearning.com -
www.loomlearning.com - Skype: intel352
*
On Tue, Nov 29, 2011 at 6:07 PM, Jonathan Langevin <
jlange...@loomlearning.com> wrote:
> I suppose I should instead stream the list of keys to client, slice keys
> in client, then fetch the objects, right?
> On Nov 29, 2011 5:21 PM, "Jonathan Langevin" <jlange...@loomlearning.com>
> wrote:
>
>> When attempting to run m/r queries that execute Riak.reduceSlice to
>> create paginated result sets, I've found an unexpected result.
>>
>> For instance, if I call Riak.reduceSlice with start = 80, end = 85, which
>> you would expect to return 5 results (knowing that you have a total of 115
>> objects stored in Riak), you might instead get 16 objects returned back.
>> This is due to the logic of Riak.reduceSlice, which just returns the
>> results-thus-far when it finds that the end value (85) is greater than the
>> current length of value (16 objects).
>>
>> I suppose ultimately this is due to how Riak sends data in chunks between
>> phases (at least, that's my understanding)? So it happens to send a small
>> chunk of 16 results from the map phase to the reduce phase, and reduce
>> returns back the current "value" chunk since 16 results is less than 85
>> results...
>>
>>
>> How can I plan for this, so that I can ensure I'm getting the expected #
>> results back, and in fair context to the "page" of results that I should be
>> viewing?
>> And just a note, while I tested with & without sorted results, the final
>> code would be with sorted results that are then paginated.
>>
>>
>> Example data payload:
>> {
>> "inputs":"demo_3_Course",
>> "query":[
>> {
>> "map":{
>> "keep":false,
>> "language":"javascript",
>> "arg":null,
>> "source":"function(value){return [value.key];}"
>> }
>> },
>> {
>> "reduce":{
>> "keep":true,
>> "language":"javascript",
>> "arg":[
>> 80,
>> 85
>> ],
>> "name":"Riak.reduceSlice"
>> }
>> }
>> ]
>> }
>> *
>>
>> <http://www.loomlearning.com/>
>> Jonathan Langevin
>> Manager, Information Technology
>> Loom Inc.
>> Wilmington, NC: (910) 241-0433 - jlange...@loomlearning.com -
>> www.loomlearning.com - Skype: intel352
>> *
>>
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
