If the problem is with net/http then we need to fix that backend, not apply hacks to the generic portions. You can verify that Riak will return valid JSON objects in each chunk by adding --raw to the curl command. Here's what I get after loading the Fast Track stock data: http://gist.github.com/399450
Sean Cribbs <s...@basho.com> Developer Advocate Basho Technologies, Inc. http://basho.com/ On May 12, 2010, at 10:55 PM, Adam Hunter wrote: > This is a confirmed bug and will be fixed soon. You can check out > http://github.com/adamhunter/ripple/blob/associations/lib/riak/bucket.rb : 65 > for a temporary hack. > > Thanks, > > Adam > > On May 12, 2010, at 10:17 PM, Andrew Harvey wrote: > >> I've just run into this one as well. >> >> >> Not sure if it's a curb thing or what, but it's making me a sad panda. We >> have 500k+ keys in our bucket, and I'm being returned chunks of invalid json >> so nothing is being yielded to my block. Needless to say, the task I'm doing >> here I'd prefer to do in a MapReduce, but unfortunately I need to get >> everything out into ruby-land for the moment. >> >> Andrew >> >> >> On 04/05/2010, at 11:46 PM, Sean Cribbs wrote: >> >>> This looks like a bug. I'll add an issue to the github tracker. >>> >>> Sean Cribbs <s...@basho.com> >>> Developer Advocate >>> Basho Technologies, Inc. >>> http://basho.com/ >>> >>> On May 4, 2010, at 9:29 AM, Adam Hunter wrote: >>> >>>> Just curb, I couldn't get net/http to work when doing chunked (something >>>> about read_body being called twice). >>>> >>>> >>>> On May 4, 2010, at 9:28 AM, Sean Cribbs wrote: >>>> >>>>> Are you using curb or net/http? >>>>> >>>>> Sean Cribbs <s...@basho.com> >>>>> Developer Advocate >>>>> Basho Technologies, Inc. >>>>> http://basho.com/ >>>>> >>>>> On May 4, 2010, at 9:26 AM, Adam Hunter wrote: >>>>> >>>>>> The problem I was running into was some of the chunks were only part of >>>>>> the json object. Decoded it would be something like this: >>>>>> >>>>>> chunk 1: {'keys':[1,2,3,4 >>>>>> chunk 2: 5,6,7,8,9]} >>>>>> >>>>>> I was running this on a bucket with about 11k keys. Using ripple I >>>>>> could only get about 150 keys back this way because all the other chunks >>>>>> were not valid json. I hacked around this for now by checking to see if >>>>>> the chunk ends in ]}, otherwise I accumulate chunks until I have a full >>>>>> json object. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Adam >>>>>> >>>>>> >>>>>> On May 4, 2010, at 9:22 AM, Sean Cribbs wrote: >>>>>> >>>>>>> This behavior is unchanged since its implementation in 0.8 or 0.9. To >>>>>>> better demonstrate what that statement means, here's the joined output >>>>>>> of a streamed keys response: >>>>>>> >>>>>>> {"props":{"name":"foo","n_val":3,"allow_mult":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10}}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]} >>>>>>> >>>>>>> If your client library supports per-chunk processing, each of those >>>>>>> {"keys":[]} objects will be an encoded chunk. The Ruby client and most >>>>>>> of the other clients support this. >>>>>>> >>>>>>> Sean Cribbs <s...@basho.com> >>>>>>> Developer Advocate >>>>>>> Basho Technologies, Inc. >>>>>>> http://basho.com/ >>>>>>> >>>>>>> On May 3, 2010, at 9:28 PM, Adam Hunter wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> I apologize if this has been covered already. >>>>>>>> >>>>>>>> In the wiki it says "If keys=stream, the response will be transferred >>>>>>>> using chunked-encoding, where each chunk is a JSON object" (on >>>>>>>> http://wiki.basho.com/display/RIAK/REST+API) >>>>>>>> >>>>>>>> This doesn't seem to be the case, sometimes a json object is broken >>>>>>>> into several chunks. Is this behavior correct? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Adam >>>>>>>> _______________________________________________ >>>>>>>> riak-users mailing list >>>>>>>> riak-users@lists.basho.com >>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>>>> >>>>>> >>>>> >>>> >>> >>> >>> >>> Click here to report this email as spam. >>> >>> _______________________________________________ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> Andrew Harvey / Developer >> lexer >> >> m/ >> t/ +61 2 9019 6379 >> w/ http://lexer.com.au >> >> Help put an end to whaling. Visit www.givewhalesavoice.com.au >> >> >> >> Please consider the environment before printing this email >> This email transmission is confidential and intended solely for the person >> or organisation to whom it is addressed. If you are not the intended >> recipient, you must not copy, distribute or disseminate the information, or >> take any action in relation to it and please delete this e-mail. Any views >> expressed in this message are those of the individual sender, except where >> the send specifically states them to be the views of any organisation or >> employer. If you have received this message in error, do not open any >> attachment but please notify the sender (above). This message has been >> checked for all known viruses powered by McAfee. >> >> For further information visit >> http://www.mcafee.com/us/threat_center/default.asp >> Please rely on your own virus check as no responsibility is taken by the >> sender for any damage rising out of any virus infection this communication >> may contain. >> >> >> This message has been scanned for malware by Websense. www.websense.com >> > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
