This is a confirmed bug and will be fixed soon. You can check out http://github.com/adamhunter/ripple/blob/associations/lib/riak/bucket.rb : 65 for a temporary hack.
Thanks, Adam On May 12, 2010, at 10:17 PM, Andrew Harvey wrote: > I've just run into this one as well. > > > Not sure if it's a curb thing or what, but it's making me a sad panda. We > have 500k+ keys in our bucket, and I'm being returned chunks of invalid json > so nothing is being yielded to my block. Needless to say, the task I'm doing > here I'd prefer to do in a MapReduce, but unfortunately I need to get > everything out into ruby-land for the moment. > > Andrew > > > On 04/05/2010, at 11:46 PM, Sean Cribbs wrote: > >> This looks like a bug. I'll add an issue to the github tracker. >> >> Sean Cribbs <s...@basho.com> >> Developer Advocate >> Basho Technologies, Inc. >> http://basho.com/ >> >> On May 4, 2010, at 9:29 AM, Adam Hunter wrote: >> >>> Just curb, I couldn't get net/http to work when doing chunked (something >>> about read_body being called twice). >>> >>> >>> On May 4, 2010, at 9:28 AM, Sean Cribbs wrote: >>> >>>> Are you using curb or net/http? >>>> >>>> Sean Cribbs <s...@basho.com> >>>> Developer Advocate >>>> Basho Technologies, Inc. >>>> http://basho.com/ >>>> >>>> On May 4, 2010, at 9:26 AM, Adam Hunter wrote: >>>> >>>>> The problem I was running into was some of the chunks were only part of >>>>> the json object. Decoded it would be something like this: >>>>> >>>>> chunk 1: {'keys':[1,2,3,4 >>>>> chunk 2: 5,6,7,8,9]} >>>>> >>>>> I was running this on a bucket with about 11k keys. Using ripple I could >>>>> only get about 150 keys back this way because all the other chunks were >>>>> not valid json. I hacked around this for now by checking to see if the >>>>> chunk ends in ]}, otherwise I accumulate chunks until I have a full json >>>>> object. >>>>> >>>>> Thanks, >>>>> >>>>> Adam >>>>> >>>>> >>>>> On May 4, 2010, at 9:22 AM, Sean Cribbs wrote: >>>>> >>>>>> This behavior is unchanged since its implementation in 0.8 or 0.9. To >>>>>> better demonstrate what that statement means, here's the joined output >>>>>> of a streamed keys response: >>>>>> >>>>>> {"props":{"name":"foo","n_val":3,"allow_mult":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10}}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]} >>>>>> >>>>>> If your client library supports per-chunk processing, each of those >>>>>> {"keys":[]} objects will be an encoded chunk. The Ruby client and most >>>>>> of the other clients support this. >>>>>> >>>>>> Sean Cribbs <s...@basho.com> >>>>>> Developer Advocate >>>>>> Basho Technologies, Inc. >>>>>> http://basho.com/ >>>>>> >>>>>> On May 3, 2010, at 9:28 PM, Adam Hunter wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> I apologize if this has been covered already. >>>>>>> >>>>>>> In the wiki it says "If keys=stream, the response will be transferred >>>>>>> using chunked-encoding, where each chunk is a JSON object" (on >>>>>>> http://wiki.basho.com/display/RIAK/REST+API) >>>>>>> >>>>>>> This doesn't seem to be the case, sometimes a json object is broken >>>>>>> into several chunks. Is this behavior correct? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Adam >>>>>>> _______________________________________________ >>>>>>> riak-users mailing list >>>>>>> riak-users@lists.basho.com >>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>>> >>>>> >>>> >>> >> >> >> >> Click here to report this email as spam. >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > Andrew Harvey / Developer > lexer > > m/ > t/ +61 2 9019 6379 > w/ http://lexer.com.au > > Help put an end to whaling. Visit www.givewhalesavoice.com.au > > > > Please consider the environment before printing this email > This email transmission is confidential and intended solely for the person or > organisation to whom it is addressed. If you are not the intended recipient, > you must not copy, distribute or disseminate the information, or take any > action in relation to it and please delete this e-mail. Any views expressed > in this message are those of the individual sender, except where the send > specifically states them to be the views of any organisation or employer. If > you have received this message in error, do not open any attachment but > please notify the sender (above). This message has been checked for all known > viruses powered by McAfee. > > For further information visit > http://www.mcafee.com/us/threat_center/default.asp > Please rely on your own virus check as no responsibility is taken by the > sender for any damage rising out of any virus infection this communication > may contain. > > > This message has been scanned for malware by Websense. www.websense.com >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
