Yes, I agree that the choice of version number can be done separately. That's why I said I'd file a separate JIRA for the documentation improvements. Having said that, there are some expectations that people have for projects that have reached 1.0.0 and we should try to allocate time for the important ones.
Ismael On 21 Jul 2017 8:07 pm, "Guozhang Wang" <wangg...@gmail.com> wrote: > Thanks Ismael. I agree with you on all these points, and for some of these > points like 3) we never have a written-down policy though in practice we > tend to follow some patterns. > > To me deciding what's the version number of the next major release does not > necessarily mean we need now to change any of these or to set the hard > rules along with it; I'd like to keep them as two separate discussions as > they seem semi-orthogonal to me. > > > Guozhang > > > On Fri, Jul 21, 2017 at 8:44 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > On the topic of documentation, we should also document which releases are > > still supported and which are not. There a few factors to consider: > > > > 1. Which branches receive bug fixes. We typically backport fixes to the > two > > most recent stable branches (the most recent stable branch typically gets > > more backports than the older one). > > > > 2. Which branches receive security fixes. This could be the same as `1`, > > but we could attempt to backport more aggressively for security fixes as > > they tend to be rare (so far at least) and the impact could be severe. > > > > 3. The release policy for stable branches. We tend to stop releasing > from a > > given stable branch before we stop backporting fixes. Maybe that's OK, > but > > it would be good to document how we decide that a bug fix release is > > needed. > > > > 4. How long are direct upgrades supported for. During the time-based > > releases discussion, we agreed to support direct upgrades for 2 years. As > > it happens, direct upgrades from 0.8.2 to 1.0.0 would not be supported if > > we follow this strictly. Not clear if we want to do that. > > > > 5. How long are older clients supported for. Current brokers support > > clients all the way to 0.8.x. > > > > 6. How long are older brokers supported for. Current clients support > 0.10.x > > and newer brokers. > > > > 7. How long are message formats supported for. We never discussed this. I > > think 5 years would probably be the minimum. > > > > Ismael > > > > P.S. I'll file a JIRA to capture this information. > > > > On Wed, Jul 19, 2017 at 10:12 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > > > Hi Stevo, > > > > > > Thanks for your feedback. We should definitely do a better job of > > > documenting things. We basically follow semantic versioning, but it's > > > currently a bit confusing because: > > > > > > 1. There are 4 segments in the version. The "0." part should be ignored > > > when deciding what is major, minor and patch at the moment, but many > > people > > > don't know this. Once we move to 1.0.0, that problem goes away. > > > > > > 2. To know what is a public API, you must check the Javadoc ( > > > https://kafka.apache.org/0110/javadoc/index.html?org/ > > > apache/kafka/clients/consumer/KafkaConsumer.html). If it's not listed > > > there, it's not public API. Ideally, it would be obvious from the > package > > > name (i.e. there would be "internals" in the name), but we are not > there > > > yet. The exception are the old Scala APIs, but they have all been > > > deprecated and they will be removed eventually (the old Scala consumers > > > won't be removed until the June 2018 release at the earliest in order > to > > > give people time to migrate). > > > > > > 3. Even though we are following reasonably common practices, we haven't > > > documented them all in one place. It would be great to do it during the > > > next release cycle. > > > > > > A few comments below. > > > > > > On Wed, Jul 19, 2017 at 1:31 AM, Stevo Slavić <ssla...@gmail.com> > wrote: > > > > > >> - APIs not labeled or labeled as stable > > >> -- change in major version is only one that can break backward > > >> compatibility (client APIs or behavior) > > >> > > > > > > To clarify, stable APIs should not be changed in an incompatible way > > > without a deprecation cycle. Independently of whether it's a major > > release > > > or not. > > > > > > > > >> -- change in minor version can introduce new features, but not break > > >> backward compatibility > > >> -- change in patch version, is for bug fixes only. > > >> > > > > > > Right, this has been the case for a while already. Also see annotations > > > below. > > > > > > > > >> - APIs labeled as evolving can be broken in backward incompatible way > in > > >> any release, but are assumed less likely to be broken compared to > > unstable > > >> APIs > > >> - APIs labeled as unstable can be broken in backward incompatible way > in > > >> any release, major, minor or patch > > >> > > > > > > The relevant annotations do explain this: > > > > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > common/annotation/ > > > InterfaceStability.html > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > common/annotation/ > > > InterfaceStability.Stable.html > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > common/annotation/ > > > InterfaceStability.Evolving.html > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > common/annotation/ > > > InterfaceStability.Unstable.html > > > > > > But we should have a section in our documentation as well. > > > > > > > > >> - deprecated stable APIs are treated as any stable APIs, they can be > > >> removed only in major release, are not allowed to be changed in > backward > > >> incompatible way in either patch or minor version release > > >> > > > > > > Right, but note that stable non-deprecated APIs provide stronger > > > guarantees in major releases (they can't be changed in an incompatible > > way). > > > > > >> > > >> This means one should be able to upgrade server and recompile/deploy > > apps > > >> with clients to new minor.patch release with dependency version change > > >> being only change needed and there would be no drama. > > >> > > > > > > That should have been the case for a while as long as you are using > > stable > > > public APIs. > > > > > >> > > >> Practice/"features" like protocol version being a parameter, and > > >> defaulting > > >> to latest so auto updated with dependency update which introduces new > > >> protocol/behavior should not be used in public client APIs. To switch > > >> between backward incompatible APIs (contract and behaviors), ideally > > user > > >> should explicitly have to change code and not dependency only, but at > > >> least > > >> it should be clearly communicated that there are breaking changes to > > >> expect > > >> even with just dependency update by e.g. giving major version release > > >> clear > > >> meaning. If app dependency on Kafka client library minor.patch on same > > >> major is updated, and if there's a change in behavior or API requiring > > app > > >> code change - it's a bug. > > >> > > > > > > Hmm, if the protocol bump provides improved behaviour, that is not a > > > backwards incompatible change though. So, I don't think I agree with > > this. > > > Of course, > > > it does mean that _downgrading_ may cause loss of functionality. That's > > > OK, in my opinion. > > > > > > Change introduced contrary to the SLO, is OK to be reported as bug. > > >> Everything else is improvement or feature request. > > >> > > >> If this was the case, and 1.0.0 was released today with APIs as they > are > > >> now, Scala client APIs even though deprecated would not break and > > require > > >> refactoring with every 1.* minor/patch release, and would only be > > allowed > > >> to be broken or removed in future major release, like 2.0.0 > > >> > > > > > > Yes, that is the plan for any _public_ Scala client APIs that are still > > > present in 1.0.0. The public Scala client APIs are the producer and > > > consumer, basically. Again, we should make this clear in our > > documentation. > > > Note that we have made an effort to keep those APIs compatible for > quite > > a > > > while. It sounds like you have had some issues, were they related to > > usage > > > of internal Admin APIs by any chance (since we didn't have a public > > > AdminClient API until very recently)? > > > > > >> > > >> It should be also clear how long is each version supported - e.g. if > > >> minor.patch had meaning that there are no backward incompatible > changes, > > >> it's OK to file a bug only for current major.minor.patch; previous > major > > >> and its last minor.patch can only have patches released up to some > time > > >> like 1 up to 3 months. > > >> > > > > > > I am not sure I understood this point correctly. Can you please > clarify? > > > > > > If there are changes in release cadence with new versioning, it should > be > > >> clear too. > > >> > > > > > > No changes are planned. We have started time-based releases less than a > > > year ago and they seem to be going well. > > > > > > Ismael > > > > > > > > > -- > -- Guozhang >
