I don't have anything specific to say wrt SASL features, but I think
this circumstance makes it clear that there are only 2 ways forward:
(1) official java client continues releasing w/ broker versioning as
an implicit compatibility test ("java client X.Y requires broker X.Y")
AND support is added to brokers so that all clients can query broker
version ("0.10.0.0") via API, enabling similar implicit compatibility
tests in non-java clients, or
(2) java client versioning is decoupled from broker versioning,
breaking reliance on implicit compatibility tests, AND all clients
forced to rely on explicit protocol compatibility tests exposed via
API (such as via KIP-35)
Is there any other way to avoid this continuing to be an issue?
-Dana
On Mon, Apr 4, 2016 at 7:14 PM, Gwen Shapira <g...@confluent.io> wrote:
> Magnus,
>
> It sounds like KIP-43 will need to change in order to support the KIP-35
> protocol. Can you add more details on what you had in mind?
>
> On Mon, Apr 4, 2016 at 12:41 PM, Magnus Edenhill <mag...@edenhill.se> wrote:
>
>> As Jun says the SASL (and SSL) handshake is not done using the Kafka
>> protocol
>> and is performed before any Kafka protocol requests pass between client and
>> server.
>>
>> It might make sense to move the SASL handshake from its custom protocol
>> format
>> into the Kafka protocol and make it use the proper Kafka protocol framing.
>>
>> (For SSL this is isnt needed since TLS has its own _standardised_
>> hand-shake format and existing SSL implementations take care of it.)
>>
>> 2016-04-04 21:20 GMT+02:00 Ismael Juma <ism...@juma.me.uk>:
>>
>> > An option would be to add a version for the handshake in the KIP-35
>> > response.
>> >
>> > Ismael
>> > On 4 Apr 2016 20:09, "Gwen Shapira" <g...@confluent.io> wrote:
>> >
>> > > I think the challenge here is that even after KIP-35 clients will not
>> > know
>> > > whether the server supports new sasl mechanisms or not, so non-Java
>> > clients
>> > > will have to assume it is not supported (and will therefore lag behind
>> on
>> > > features).
>> > >
>> > > I think this highlights a short-coming of KIP-35, and I'm wondering if
>> > > there are good ways to address this.
>> > >
>> > > Gwen
>> > >
>> > >
>> > > On Mon, Apr 4, 2016 at 12:05 PM, Jun Rao <j...@confluent.io> wrote:
>> > >
>> > > > I think with KIP-43, the existing way of sasl handshake during
>> > connection
>> > > > still works. It's just that if you want to support non-GSSAPI, you
>> will
>> > > > need a new sasl handshake implementation in the client. It's
>> > unfortunate
>> > > > that Protocol currently only covers the communication after the
>> > > connection
>> > > > is ready to use, but not during handshake. For now, we can probably
>> > just
>> > > > document this change during handshake since changing the
>> implementation
>> > > is
>> > > > optional.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Jun
>> > > >
>> > > > On Mon, Apr 4, 2016 at 11:28 AM, Gwen Shapira <g...@confluent.io>
>> > wrote:
>> > > >
>> > > > > Hi Kafka Team,
>> > > > >
>> > > > > As a practical test-case of KIP-35, I'd like to turn your attention
>> > to
>> > > > > KIP-43:
>> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-43
>> > > > >
>> > > > > KIP-43 makes an interesting modification to the protocol, but only
>> > > under
>> > > > > specific conditions:
>> > > > >
>> > > > > "*Client flow:*
>> > > > >
>> > > > > 1. If sasl.mechanism is not GSSAPI, send a packet with the
>> > mechanism
>> > > > > name to the server. Otherwise go to Step 3.
>> > > > > - Packet Format: | Version (Int16) | Mechanism (String) |
>> > > > > 2. Wait for response from the server. If the error code in the
>> > > > response
>> > > > > is non-zero, indicating failure, report the error and fail
>> > > > > authentication.
>> > > > > 3. Perform SASL authentication with the configured client
>> > mechanism
>> > > > >
>> > > > > *Server flow:*
>> > > > >
>> > > > > 1. Wait for first authentication packet from client
>> > > > > 2. If this packet is a not valid mechanism request, go to Step 4
>> > and
>> > > > > process this packet as the first GSSAPI client token
>> > > > > 3. If the client mechanism received in Step 2 is enabled in the
>> > > > broker,
>> > > > > send a response with error code zero and start authentication
>> > using
>> > > > the
>> > > > > specified mechanism. Otherwise, send an error response including
>> > the
>> > > > > list
>> > > > > of enabled mechanisms and fail authentication.
>> > > > > - Packet Format: | ErrorCode (Int16) | EnabledMechanisms
>> > > > > (ArrayOf(String))
>> > > > > |
>> > > > > 4. Perform SASL authentication with the selected mechanism. If
>> > > > mechanism
>> > > > > exchange was skipped, process the initial packet that was
>> received
>> > > > from
>> > > > > the
>> > > > > client first."
>> > > > >
>> > > > >
>> > > > > I'd love to know how this will be communicated to clients via
>> > > > > VersionRequest proposed in KIP-35 (mostly because Jun and I need to
>> > > > review
>> > > > > KIP-43 and we want to be sure we are not breaking the new protocol
>> at
>> > > the
>> > > > > same time we introduce it)
>> > > > >
>> > > > > Do we:
>> > > > > 1. Bump protocol version of every single Request? Even though
>> unless
>> > > you
>> > > > > are using a new sasl mechanism nothing changes?
>> > > > > 2. Ignore and not bump protocol? If so, how will clients know that
>> > new
>> > > > > sasl.mechanisms are supported?
>> > > > > 3. Something else?
>> > > > >
>> > > > > Gwen
>> > > > >
>> > > >
>> > >
>> >
>>
