Also we are trying to document all this stuff well with the 0.8 release. What could I do to improve this document: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
Its kind of hard to write good documentation, so any feedback would be appreciated. -Jay On Thu, Dec 6, 2012 at 9:20 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > Hi Ben, > > If I understand your question, you are asking if it is possible to > decouple or pipeline the sending of the request and the reading of the > response. E.g. do something like > send(req1) > send(req2) > recv(resp1) > recv(resp2) > I think your question is, if I do this how do I know what response goes to > what request. > > The answer is that the server guarantees in-order processing of requests > on a given connection. So correlating requests to responses can be done by > keeping a queue of as-yet unanswered requests. In 0.8 we also added an > official correlation id (an integer field you can set that the server > passes back unchanged in the response. This is meant to help do tracing > across client and server and may help to debug your implementation. We do > not currently send back the API key since there is only one correct value > that could have--whatever the api of the corresponding request was. > > One caveat is that the server doesn't really fully pipeline request > processing at this time so if you do pipeline you may not get quite as much > parallelism as you expect (you can never get full parallelism because of > the ordering guarantee). But in the current code until a response is sent > for req1 the server won't start reading req2. This is not hard to fix and > we will get to it in the next major release I think. > > -Jay > > > > On Thu, Dec 6, 2012 at 7:50 AM, ben fleis <b...@monkey.org> wrote: > >> I just realized something important. The BoundedByteBufferSend comment >> above is not in sync with my original question. I was asking about >> Responses -- meaning, can I multiplex off a single socket, and when >> reading >> an arbitrary response, switch on request_id to the right handler? >> >> In the blocking console sync client (which is the only trivial way to >> follow the code that I've found -- please enlighten if there's an async >> thing I can easily trace), this is a non issue, since you request, wait, >> parse expected response. >> >> Or do I need instead to just open separate sockets? (Not a big deal, but >> nice to understand clearly.) >> >> ben >> > >
