I've been struggling with this for a few days now. I have a pair of related
Map/Reduce queries. Executed individually they produce the expected
results, but when executed in parallel (simultaneous XHR calls from the
browser), their results tend to get mixed up with each other; I see values
from both mixed together in a single result.
Once that corruption happens, even the calls executed individually can end
up with cross-pollinated results. It seems to be related to the caching of
the anonymous javascript functions used, as adding a space to the function
declaration clears the issue up. Taking that space back out causes the
problem again.
I can confirm that the corruption can become persistent by using a line such
as (PHP client):
ejsLog('/tmp/mapred/reduce_" . join('.', explode('\\', get_class($this))) .
".log', JSON.stringify(values));
After the corruption happens, if I clear the log path, execute just the one
map/red operation, I will see log entries appear for both classes (the class
name being encoded as part of the function declaration). The problem seems
to happen for both map and reduce operations, though I have not specifically
noticed cross-contamination *between* map and reduce functions, just
map-for-map and reduce-for-reduce.
Adding this as the first line of the function declaration seems to solve the
problem, but this is obviously something intended to defeat caching, so I'm
sure there's a performance cost to this (note, just get_class($this) is not
sufficient, cross-contamination can still happen, each call needs to be *
unique* call-to-call, thus the mt_rand()).
"//".get_class($this).' '.mt_rand()."
I noticed Francisco Treacy had this same problem back in September:
http://riak-users.197444.n3.nabble.com/reduce-headaches-td1524860.html but I
didn't see any resolution there. Is there a potential longer term cost to
my cachebusting, such as memory exhaustion (i.e. is there any garbage
collection on the function cache)?
Any help would be sincerely appreciated!
-Eric
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
