We (Jay + me) had some extra information we wanted to see in the KIP before we are comfortable voting:
* Where does the Java client fits in. Hopefully we can use this KIP to standardize behavior and guarantees between Java and non-Java clients, so when we reason about the Java clients, which most Kafka developers are familiar with, we will make the right decisions for all clients. * When do we bump the protocol? I think 90% of the issue is not that the version got bumped but rather that we changed behavior without bumping versions. For the new VersionRequest to be useful, we need to all know when to get new versions... * How do we test / validate - I think our recent experience shows that our protocol tests and compatibility tests are still inadequate. Having VersionRequest is useless if we can't validate that Kafka actually implements the protocol it says it does (and we caught this breaks twice in the last two weeks) * Error handling of protocol mismatches Ashish kindly agreed to think about this and improve the KIP. We'll resume the vote as soon as he's back :) Gwen On Wed, Mar 23, 2016 at 5:55 PM, Dana Powers <dana.pow...@gmail.com> wrote: > speaking of pending KIPs, what's the status on this one? > > > On Fri, Mar 18, 2016 at 9:47 PM, Ashish Singh <asi...@cloudera.com> wrote: > > > Hey Jay, > > > > Answers inline. > > > > On Fri, Mar 18, 2016 at 10:45 AM, Jay Kreps <j...@confluent.io> wrote: > > > > Hey Ashish, > > > > > > Couple quick things: > > > > > > 1. You list as a rejected alternative "making the documentation the > > > source of truth for the protocol", but I think what you actually > > > describe in that section is global versioning, which of those two > > > things are we voting to reject? I think this is a philosophical point > > > but an important one... > > > > > One of the major differences between Option 3 and other options discussed > > on KIP is that Option 3 is documentation oriented and it is that what I > > wanted to capture in the title. I am happy to change it to global > > versioning. > > > > > > > 2. Can you describe the changes necessary and classes we'd have to > > > update in the java clients to make use of this feature? What would > > > that look like? One concern I have is just the complexity necessary to > > > do the per-connection protocol version check and really handle all the > > > cases. I assume you've thought through what that looks like, can you > > > sketch that out for people? > > > > > I would imagine any client, even Java client, would follow the steps > > mentioned here > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version#KIP-35-Retrievingprotocolversion-Aclientdeveloperwantstoaddsupportforanewfeature.1 > > >. > > Below are my thoughts on how java client can maintain api versions > > supported by various brokers in cluster. > > > > 1. ClusterConnectionStates can provide info on whether api versions > have > > been retrieved for a connection or not. > > 2. NetworkClient.handleConnections can send ApiVersionQueryRequest to > > newly connected nodes. > > 3. NetworkClient can be enhanced to handle ApiVersionQueryResponse and > > set ClusterConnectionStates to indicate api versions have been > retrieved > > for the node. > > 4. NetworkClient maintains mapping Node -> [(api_key, min_ver, > > max_ver)], brokerApiVersions, cached. > > 5. NetworkClient.processDisconnection can remove entry for a node from > > brokerApiVersions cache. > > 6. NetworkClient.canSendRequest can have an added condition on node to > > have api versions available. > > > > With the above changes, at any given point of time NetworkClient will be > > aware of api versions supported by each of the connected nodes. I am not > > sure if the above changes are the best way to do it, people are welcome > to > > pitch in. Does it help? > > > > > > > -Jay > > > > > > On Mon, Mar 14, 2016 at 3:54 PM, Ashish Singh <asi...@cloudera.com> > > wrote: > > > > Hey Guys, > > > > > > > > I would like to start voting process for *KIP-35: Retrieving protocol > > > > version*. The KIP is available here > > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version > > > >. > > > > Here > > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version#KIP-35-Retrievingprotocolversion-SummaryofthechangesproposedaspartofthisKIP > > > > > > > > is a brief summary of the KIP. > > > > > > > > The vote will run for 72 hours. > > > > > > > > -- > > > > > > > > Regards, > > > > Ashish > > > > > > > -- > > > > Regards, > > Ashish > > >
