I agree in a "perfect is the enemy of good" sense. I don't think what's expressed in the API is necessarily the *ideal* thing for client developers, which is who I think this API should be optimizing for. That said, this is a practical solution to the problem and while finer-grained than I think is actually needed, it's better to be too fine grained and require a bit of extra effort for client developers than to be too coarse-grained (e.g., global version numbers) and make it difficult/impossible to get the desired behavior.
Short version: if Magnus and Dana think this is a reasonable solution, I'll sign off on it since I see nothing glaringly wrong or bad about it and it solves real problems for them :) I think we arrived at a policy for Java client compatibility which works fine when everything is developed in lockstep in the same codebase, but isn't actually practical with other clients. In practice, no client developers follow the same compatibility matrix -- they want clients compiled with versions more recently than brokers to continue to work with brokers. Although I understand how you can make the approach used with Java clients work, I also understand why folks would want this other approach to work. Given that a lot of clients want a different compatibility matrix than the Java clients support and this KIP will enable that (even if it requires some translations in the client libraries to figure out what features are supported based on API versions), I think we should adopt this support. -Ewen On Thu, Mar 31, 2016 at 3:05 PM, Ashish Singh <asi...@cloudera.com> wrote: > I agree with Jason. The blocking factor has been how to use the proposed > changes in java client without making it super complicated. With Jason's > suggestion we can get past this blocker, while keeping the core of the > proposal intact. > > > On Thu, Mar 31, 2016 at 2:51 PM, Jason Gustafson <ja...@confluent.io> > wrote: > > > Bumping this thread. I talked with Ashish and Magnus about this KIP > offline > > and I'm gradually coming over. The new API actually stands by itself > > outside of the discussion about whether the client should support > backwards > > compatibility or not. For the Java client, we could continue to support > the > > current compatibility approach in which the client supports only brokers > > with the same version or greater. In that case, we would use this API > only > > to assert that the current API versions are all supported, and raise an > > exception if they are not. This gives us the capability going forward to > > detect when the client is talking to an older broker, which we don't have > > right now. This check should be straightforward, so we could do it now, > > which would resolve some of the uneasiness about having an unused feature > > which we depended on other clients to test for us. Does that make sense > or > > not? > > > > -Jason > > > > On Thu, Mar 17, 2016 at 4:06 PM, Ashish Singh <asi...@cloudera.com> > wrote: > > > > > We have proposed and discussed majorly three approaches so far, there > > were > > > many minor versions with small variations. Comparing them really > > requires a > > > side by side proposal and their pros/cons, and I agree with others that > > > this has been lacking in the KIP. We just updated the KIP with > following > > > details. > > > > > > 1. Provide proposed changes in all the three proposals we have > discussed > > so > > > far. Except the current proposal, these proposals are in rejected > > > alternatives. > > > 2. Provide reasoning on why the rejected proposals were rejected. > > > 3. Add scenarios for all of these proposals from a client developer and > > > core Kafka developer point of view. > > > > > > As we are really close to 0.10 deadline, a quick round of voting will > > > really help. If you really do not like the idea, please feel free to > say > > > so. If the vote fails for the current proposal, it can at lease provide > > > recommendations that we should consider for next version of proposal > and > > > put it up for vote again for next release. However, as stated earlier > by > > > multiple people having this ASAP will be awesome. > > > > > > On Thu, Mar 17, 2016 at 3:29 PM, Dana Powers <dana.pow...@gmail.com> > > > wrote: > > > > > > > On Thu, Mar 17, 2016 at 1:42 PM, Gwen Shapira <g...@confluent.io> > > wrote: > > > > > > > > > "I think I would make this approach work by looking at the released > > > > server > > > > > version documentation for each version that I am trying to support > > and > > > > test > > > > > against*, manually identify the expected "protocol vectors" each > > > > supports, > > > > > store that as a map of vectors to "broker versions", check each > > vector > > > at > > > > > runtime until I find a match, and write code compatibility checks > > from > > > > > there." > > > > > > > > > > How is this better than a global version ID? > > > > > > > > > > > > As a client developer, it seems roughly the same. I think it probably > > > > avoids the server development workflow issues, and possibly the need > to > > > > agree on semantics of the global version ID? But others surely are > more > > > > qualified than me to comment on that part. > > > > > > > > -Dana > > > > > > > > > > > > > > > > -- > > > > > > Regards, > > > Ashish > > > > > > > > > -- > > Regards, > Ashish > -- Thanks, Ewen
