Sylvain,

Bob is indeed correct. This list contains the character 153 which is a control 
character in Unicode, hence mochijson:decode() giving an error, causing the 
MapReduce process to crash. I couldn't reproduce a complete unavailability 
locally though, Riak (also 0.14.2) didn't seem to be bothered by that crash, 
the process was simply restarted. If you could send me some logs to peek at I 
could investigate the issue further.

You need to make sure that the string you're storing in Riak is properly 
encoded for Unicode (i.e. resulting in \u1234 character sets) if your 
application and MapReduce code rely on that. If you need to enforce this before 
storing data inside Riak, you can use a pre-commit hook to validate the JSON 
using e.g. mochijson2:decode() and have your application respond accordingly, 
see [1] for an example.

[1] https://gist.github.com/1031311

Mathias Meyer
Developer Advocate, Basho Technologies

On Freitag, 17. Juni 2011 at 03:09, Sylvain Niles wrote:
> So we've been encountering some input that has json which is parsed
> just fine by ruby but causes riak map reduce to die a horrible death.
> 
> Running Riak 0.14.2 with json2.js updated to the latest from github
> (thanks for that suggestion Dan)
> 
> Steps to reproduce:
> 
> bin/riak attach
> 
> Badstuff = <<123,34,97,100,100,114,101,115,
>  115,34,58,34,84,117,100,111,
>  114,32,80,108,97,99,101,32,72,
>  105,115,116,111,114,105,99,32,
>  72,111,117,115,101,32,97,110,
>  100,32,71,97,114,100,101,110,
>  44,32,49,54,52,52,32,51,49,115,
>  116,32,83,116,114,101,101,116,
>  44,32,78,87,44,32,87,97,115,
>  104,105,110,103,116,111,110,34,
>  44,34,112,114,105,99,101,34,58,
>  56,46,51,56,44,34,100,117,114,
>  97,116,105,111,110,34,58,55,50,
>  48,48,46,48,44,34,99,104,97,
>  114,105,116,121,34,58,110,117,
>  108,108,44,34,116,104,105,115,
>  95,119,101,101,107,34,58,116,
>  114,117,101,44,34,116,111,100,
>  97,121,34,58,102,97,108,115,
>  101,44,34,99,114,101,97,116,
>  101,100,95,97,116,34,58,34,84,
>  104,117,32,74,117,110,32,49,54,
>  32,49,53,58,49,49,58,49,54,32,
>  45,48,53,48,48,32,50,48,49,49,
>  34,44,34,116,105,116,108,101,
>  34,58,34,70,97,116,104,101,114,
>  153,115,32,68,97,121,32,71,105,
>  108,100,105,110,103,32,87,111,
>  114,107,115,104,111,112,34,44,
>  34,99,117,108,116,117,114,101,
>  100,34,58,110,117,108,108,44,
>  34,117,112,100,97,116,101,100,
>  95,97,116,34,58,34,84,104,117,
>  32,74,117,110,32,49,54,32,49,
>  53,58,49,49,58,49,55,32,45,48,
>  53,48,48,32,50,48,49,49,34,44,
>  34,114,101,99,117,114,114,105,
>  110,103,95,111,110,34,58,110,
>  117,108,108,44,34,117,115,101,
>  114,110,97,109,101,34,58,34,
>  101,118,101,110,116,98,114,105,
>  116,101,34,44,34,117,114,108,
>  34,58,34,104,116,116,112,58,47,
>  47,116,117,100,111,114,112,108,
>  97,99,101,102,97,116,104,101,
>  114,115,100,97,121,46,101,118,
>  101,110,116,98,114,105,116,101,
>  46,99,111,109,34,44,34,108,101,
>  97,114,110,34,58,110,117,108,
>  108,44,34,95,116,121,112,101,
>  34,58,34,69,118,101,110,116,34,
>  44,34,116,97,103,115,34,58,91,
>  34,102,97,116,104,101,114,39,
>  115,32,100,97,121,34,44,34,32,
>  103,105,108,100,105,110,103,34,
>  44,34,32,97,114,116,115,32,38,
>  32,99,114,97,102,116,115,34,44,
>  34,32,111,116,104,101,114,34,
>  93,44,34,108,110,103,34,58,45,
>  55,55,46,48,54,50,53,56,49,44,
>  34,100,114,105,110,107,115,34,
>  58,110,117,108,108,44,34,110,
>  101,120,116,95,119,101,101,107,
>  34,58,102,97,108,115,101,44,34,
>  102,97,109,105,108,121,34,58,
>  110,117,108,108,44,34,99,104,
>  101,97,112,34,58,110,117,108,
>  108,44,34,114,101,99,117,114,
>  114,105,110,103,95,117,110,116,
>  105,108,34,58,110,117,108,108,
>  44,34,116,111,109,111,114,114,
>  111,119,34,58,102,97,108,115,
>  101,44,34,116,111,110,105,103,
>  104,116,34,58,102,97,108,115,
>  101,44,34,116,104,105,115,95,
>  119,101,101,107,101,110,100,34,
>  58,116,114,117,101,44,34,116,
>  105,109,101,95,122,111,110,101,
>  34,58,45,49,56,48,48,48,46,48,
>  44,34,100,101,115,99,114,105,
>  112,116,105,111,110,34,58,34,
>  83,97,116,117,114,100,97,121,
>  44,32,74,117,110,101,32,49,56,
>  32,124,32,49,48,58,51,48,32,97,
>  46,109,46,32,32,92,114,92,110,
>  32,67,101,108,101,98,114,97,
>  116,101,32,70,97,116,104,101,
>  114,226,128,153,115,32,68,97,
>  121,32,119,105,116,104,32,65,
>  109,101,114,105,99,97,226,128,
>  153,115,32,70,111,117,110,100,
>  105,110,103,32,70,97,116,104,
>  101,114,44,32,71,101,111,114,
>  103,101,32,87,97,115,104,105,
>  110,103,116,111,110,33,32,70,
>  97,109,105,108,105,101,115,32,
>  101,110,106,111,121,32,97,110,
>  32,105,110,116,101,114,97,99,
>  116,105,118,101,32,116,111,117,
>  114,32,116,104,114,111,117,103,
>  104,32,116,104,101,32,104,105,
>  115,116,111,114,105,99,32,109,
>  97,110,115,105,111,110,32,116,
>  111,32,117,110,99,111,118,101,
>  114,32,116,114,101,97,115,117,
>  114,101,115,32,102,114,111,109,
>  32,71,101,111,114,103,101,32,
>  87,97,115,104,105,110,103,116,
>  111,110,32,97,110,100,32,108,
>  101,97,114,110,32,97,98,111,
>  117,116,32,116,104,101,32,104,
>  105,115,116,111,114,121,32,111,
>  102,32,103,105,108,100,105,110,
>  103,46,32,65,102,116,101,114,
>  32,116,104,101,32,116,111,117,
>  114,44,32,102,97,109,105,108,
>  105,101,115,32,119,111,114,107,
>  32,116,111,103,101,116,104,101,
>  114,32,116,111,32,103,105,108,
>  100,32,97,32,102,114,97,109,
>  101,32,119,105,116,104,32,103,
>  111,108,100,32,108,101,97,102,
>  32,116,111,32,116,97,107,101,
>  32,104,111,109,101,46,34,44,34,
>  108,97,116,34,58,51,56,46,57,
>  49,49,55,48,57,44,34,101,110,
>  116,101,114,116,97,105,110,101,
>  100,34,58,110,117,108,108,44,
>  34,114,101,99,117,114,114,105,
>  110,103,34,58,110,117,108,108,
>  44,34,116,111,109,111,114,114,
>  111,119,95,110,105,103,104,116,
>  34,58,102,97,108,115,101,44,34,
>  100,105,115,112,108,97,121,95,
>  116,105,109,101,34,58,34,84,
>  104,105,115,32,83,97,116,117,
>  114,100,97,121,32,97,116,32,49,
>  48,58,51,48,65,77,34,44,34,115,
>  116,97,114,116,95,116,105,109,
>  101,34,58,49,51,48,56,52,48,55,
>  52,48,48,46,48,44,34,114,101,
>  99,117,114,114,105,110,103,95,
>  116,121,112,101,34,58,110,117,
>  108,108,125>>.
> 
> {ok, C} = riak:local_client().
> RObj = riak_object:new(<<"badutf">>, <<"badutf">>, Badstuff).
> C:put(RObj).
> 
> 
> Ripple console:
> 
>  badutf = Riak::MapReduce.new(Ripple.client).add('badutf').map("function(v){
> return [JSON.parse(v.values[0])]; }", :keep => true).run
> 
> This kills riak in a very bad way, have to shut it down, delete the
> mr_queue, then restart. Then the bucket is off limits until you find
> the offending key and delete it (which is very hard to do if you can't
> run a map).
> 
> 
> Any ideas?
> 
> Thanks,
> Sylvain
> 
> _______________________________________________
> riak-users mailing list
> riak-users@lists.basho.com (mailto:riak-users@lists.basho.com)
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Reply via email to