Ah thanks, I didn't know about the keys=false option. I eventually found the key streaming feature an built it into my rial wrapper. The weird problems with link headers went away.
On Monday, May 10, 2010, Sean Cribbs <s...@basho.com> wrote: > Rick, > > If you don't want to list the keys when retrieving the bucket properties, > pass the keys=false query string. When you have lots of keys, it's best to > use keys=stream to retrieve them, which will prevent Riak from building a > huge JSON object in memory. Instead, each chunk in the stream will be a JSON > object containing a number of keys. > > Sean Cribbs <s...@basho.com> > Developer Advocate > Basho Technologies, Inc. > http://basho.com/ > > On May 10, 2010, at 3:29 AM, Rick Olson wrote: > >> Hey guys, I enjoyed going through the Riak Fast Track, so I wrote a >> node.js library for it: >> >> http://techno-weenie.net/2010/5/10/rhodes-riak-on-node-js >> >> I ran into one problem though: node.js choked on the Link headers when >> getting a bucket's properties. I realize this is a node.js bug, but I >> found it kind of odd: >> >> $ curl http://127.0.0.1:8091/riak/goog -i >> HTTP/1.1 200 OK >> Vary: Accept-Encoding >> Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) >> Link: </riak/goog/2006-05-22>; riaktag="contained", >> </riak/goog/2006-05-04>; riaktag="contained", ETC >> >> If there are 1400 links, node.js has problems. If there are several >> thousand, even curl starts having problems: >> >> curl: (27) Avoided giant realloc for header (max is 102400)! >> >> So, if I grab the contents of a bucket to get the list of keys, it >> also sends a giant Link header pointing to each key? The Link header >> ends up being much larger than the response body. Is it really >> necessary to point to each resource, when the body contains an array >> of keys? >> >> At any rate, here's how the map reduce API looks while implementing >> the sample functions from the fast track: >> >> http://github.com/technoweenie/rhodes/blob/master/examples/fast-track/mapred.js#L6-19 >> >> -- >> Rick Olson >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > -- Rick Olson _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
