This makes sense to me. Now I shall use this knowledge to take over the world. Or to re-write part of the CorrugatedIron MapReduce functionality. --- Jeremiah Peschka - Founder, Brent Ozar PLF, LLC Microsoft SQL Server MVP
On Sep 26, 2011, at 6:51 AM, Bryan Fink wrote: > On Sat, Sep 24, 2011 at 4:39 PM, Jeremiah Peschka > <jeremiah.pesc...@gmail.com> wrote: >> MapReduce phases with a keep set to true are returning in multiple >> RpbMapRedResp messages. I expected this, especially since hte API says it's >> possible. What's causing the issue is the output format. > … >> Is there a reason that Riak is returning discrete JSON chunks instead of >> streaming back raw bytes to the client? Is there less documented change in >> Riak 1.0 that will prevent this type of behavior (which incidentally didn't >> crop up until the 1.0 pre-release builds). > > Hi, Jermiah. We recently found a similar misunderstanding in the Java > client. You read the API correctly, but the pre-1.0 MapReduce system > never exploited that feature, so clients implementing that behavior > didn't have much to test against. > > The intent of returning discrete JSON chunks is to allow each response > chunk to be fully consumable without further information. You should > be able to parse each response independently, instead of having to > wait for a "done" signal. This encapsulation is also important when > multiple phases set keep=true, because it allows them to interleave > results in the response, instead of blocking on each other. > > The HTTP interface uses a similar setup when streamed results are > requested, using the query parameter chunked=true. Since all PB > results are "streamed", the same encapsulation is used. > > -Bryan _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
