> I think #1 may be tricky in practice. The response format is always
> dictated by the request format so how do I know if the bytes I got back are
> a valid response to the given request or are the UnknownRequestResponse?
On the other hand, from the client developer perspective, having to
figure out that you are looking at a closed socket because you tried
to use an API that wasn't implemented in a specific version can be
pretty annoying.
Another way to do it is to move error_code field (currently
implemented in pretty much every single Response schema) to the
Response Header, and then we could use it for "meta errors" such as
UnknownAPI.
Its a much bigger change than adding a new Request type, but possibly worth it?
>
> #2 would be a good fix for the problem I think. This might be a good
> replacement for the echo api and would probably serve the same purpose
> (checking if the server is alive).
>
> #3 is a little dangerous because we actually want clients to only pay
> attention to the protocol versions which are per-api, not the server
> version. I.e. we actually don't want the client to do something like check
> serverVersion.equals("0.8.2") because we want to be able to release the
> server at will and have it keep answering protocols in a backwards
> compatible way. I.e. a client that uses just metadata request and produce
> request should only care about the version of these protocols it implements
> being supported not about the version of the server or the version of any
> other protocol it doesn't use. This is the rationale behind versioning the
> apis independently rather than having a single protocol version that we
> would have to bump every time an internal broker-broker protocol changed.
>
> -Jay
>
> On Mon, Jan 12, 2015 at 1:32 PM, Gwen Shapira <gshap...@cloudera.com> wrote:
>
>> We ran into similar difficulties, both when trying to get Kafka to use
>> new APIs when available and when testing the wire protocol.
>>
>> +1 for all three suggestions.
>>
>> #1 sounds like the bare minimum, but I'm not sure how much it will
>> complicate the clients (now we expect either a response or an Unknown
>> message and need to be able to distinguish between them from the byte
>> array).
>>
>> #2 and #3 both makes lots of sense.
>>
>> Gwen
>>
>>
>> On Mon, Jan 12, 2015 at 1:15 PM, Dana Powers <dana.pow...@rd.io> wrote:
>> > Hi all -- continuing on the compatibility discussion:
>> >
>> > I've found that it is very difficult to identify when a server does not
>> > recognize an api
>> > (I'm using kafka-python to submit wire-protocol requests). For example,
>> > when I
>> > send a ConsumerMetadataRequest to an 0.8.1.1 server, I get a closed
>> socket
>> > *[stacktrace below]. The server raises an error internally, but does not
>> > send any
>> > meaningful response. I'm not sure whether this is the intended behavior,
>> > but
>> > maintaining clients in an ecosystem of multiple server versions with
>> > different API
>> > support it would be great to have a way to determine what the server
>> > supports
>> > and what it does not.
>> >
>> > Some suggestions:
>> >
>> > (1) An UnknownAPIResponse that is returned for any API or API Version
>> > request
>> > that is unsupported.
>> >
>> > (2) A server metadata API to get the list of supported APIs and/or API
>> > versions supported.
>> >
>> > (3) A server metadata API to get the published version of the server
>> (0.8.2
>> > v. 0.8.1.1, etc).
>> >
>> >
>> > Thoughts?
>> >
>> >
>> > Dana Powers
>> > Rdio, Inc.
>> > dana.pow...@rd.io
>> > rdio.com/people/dpkp/
>> >
>> > *stacktrace:
>> > ```
>> > [2015-01-12 13:03:55,719] ERROR Closing socket for /127.0.0.1 because of
>> > error (kafka.network.Processor)
>> > kafka.common.KafkaException: Wrong request type 10
>> > at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
>> > at
>> kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:53)
>> > at kafka.network.Processor.read(SocketServer.scala:353)
>> > at kafka.network.Processor.run(SocketServer.scala:245)
>> > at java.lang.Thread.run(Thread.java:722)
>> > ```
>>
