I am in the process of updating my erlang producer/consumer client to 0.8.
Being lazy, I don't really want to go to Scala code. So I am just going
through
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.<https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol>Most
things are clear and clearer with discussion in this thread.
However a few queries:
RequestObject
(a) What are the specific values for the different requests in the
ApiKey?
(b) What are the specific legal values for ApiVersions? I assume that
this might change for future versions of kafka.
ResponseObject
(a) Would there be a difference between ResponseVersion and the
ApiVersions of the RequestObject? Would be possible, in the future, to
request ApiVersion of X and get a ResponseVersion of Y? How would the
cliient know how to parse ResponseVersion of Y. This makes me think that I
don't understand the motivation behind ApiVersions. Can someone shed some
light?
Thanks
Milind
On Thu, Dec 6, 2012 at 9:21 AM, Jay Kreps <jay.kr...@gmail.com> wrote:
> Also we are trying to document all this stuff well with the 0.8 release.
> What could I do to improve this document:
>
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
>
> Its kind of hard to write good documentation, so any feedback would be
> appreciated.
>
> -Jay
>
>
> On Thu, Dec 6, 2012 at 9:20 AM, Jay Kreps <jay.kr...@gmail.com> wrote:
>
> > Hi Ben,
> >
> > If I understand your question, you are asking if it is possible to
> > decouple or pipeline the sending of the request and the reading of the
> > response. E.g. do something like
> > send(req1)
> > send(req2)
> > recv(resp1)
> > recv(resp2)
> > I think your question is, if I do this how do I know what response goes
> to
> > what request.
> >
> > The answer is that the server guarantees in-order processing of requests
> > on a given connection. So correlating requests to responses can be done
> by
> > keeping a queue of as-yet unanswered requests. In 0.8 we also added an
> > official correlation id (an integer field you can set that the server
> > passes back unchanged in the response. This is meant to help do tracing
> > across client and server and may help to debug your implementation. We do
> > not currently send back the API key since there is only one correct value
> > that could have--whatever the api of the corresponding request was.
> >
> > One caveat is that the server doesn't really fully pipeline request
> > processing at this time so if you do pipeline you may not get quite as
> much
> > parallelism as you expect (you can never get full parallelism because of
> > the ordering guarantee). But in the current code until a response is sent
> > for req1 the server won't start reading req2. This is not hard to fix and
> > we will get to it in the next major release I think.
> >
> > -Jay
> >
> >
> >
> > On Thu, Dec 6, 2012 at 7:50 AM, ben fleis <b...@monkey.org> wrote:
> >
> >> I just realized something important. The BoundedByteBufferSend comment
> >> above is not in sync with my original question. I was asking about
> >> Responses -- meaning, can I multiplex off a single socket, and when
> >> reading
> >> an arbitrary response, switch on request_id to the right handler?
> >>
> >> In the blocking console sync client (which is the only trivial way to
> >> follow the code that I've found -- please enlighten if there's an async
> >> thing I can easily trace), this is a non issue, since you request, wait,
> >> parse expected response.
> >>
> >> Or do I need instead to just open separate sockets? (Not a big deal,
> but
> >> nice to understand clearly.)
> >>
> >> ben
> >>
> >
> >
>
