+1 on 1.0. I think it makes a lot of sense given the point Kafka is now. To me Kafka has absolutely reached 1.0 in terms of features/functions. That said, I do share the same opinion with others that the usablity and policies might still need some improvement to meet the 1.0 standard.
On Sat, Jul 22, 2017 at 7:21 AM, Matthias J. Sax <matth...@confluent.io> wrote: > I agree with Ismael. If we go for 1.0 (what I do support), we need to > meet people's expectations and should document all policies and > guarantees explicitly. We might also consider to support older releases > longer and do bug fix release for older releases, too. > > Other projects (like Flink) do a fantastic job with this regard and we > should learn from them. > > -Matthias > > On 7/21/17 9:50 PM, Guozhang Wang wrote: > > That's fair enough too. > > > > Guozhang > > > > On Fri, Jul 21, 2017 at 12:13 PM, Ismael Juma <isma...@gmail.com> wrote: > > > >> Yes, I agree that the choice of version number can be done separately. > >> That's why I said I'd file a separate JIRA for the documentation > >> improvements. Having said that, there are some expectations that people > >> have for projects that have reached 1.0.0 and we should try to allocate > >> time for the important ones. > >> > >> Ismael > >> > >> On 21 Jul 2017 8:07 pm, "Guozhang Wang" <wangg...@gmail.com> wrote: > >> > >>> Thanks Ismael. I agree with you on all these points, and for some of > >> these > >>> points like 3) we never have a written-down policy though in practice > we > >>> tend to follow some patterns. > >>> > >>> To me deciding what's the version number of the next major release does > >> not > >>> necessarily mean we need now to change any of these or to set the hard > >>> rules along with it; I'd like to keep them as two separate discussions > as > >>> they seem semi-orthogonal to me. > >>> > >>> > >>> Guozhang > >>> > >>> > >>> On Fri, Jul 21, 2017 at 8:44 AM, Ismael Juma <ism...@juma.me.uk> > wrote: > >>> > >>>> On the topic of documentation, we should also document which releases > >> are > >>>> still supported and which are not. There a few factors to consider: > >>>> > >>>> 1. Which branches receive bug fixes. We typically backport fixes to > the > >>> two > >>>> most recent stable branches (the most recent stable branch typically > >> gets > >>>> more backports than the older one). > >>>> > >>>> 2. Which branches receive security fixes. This could be the same as > >> `1`, > >>>> but we could attempt to backport more aggressively for security fixes > >> as > >>>> they tend to be rare (so far at least) and the impact could be severe. > >>>> > >>>> 3. The release policy for stable branches. We tend to stop releasing > >>> from a > >>>> given stable branch before we stop backporting fixes. Maybe that's OK, > >>> but > >>>> it would be good to document how we decide that a bug fix release is > >>>> needed. > >>>> > >>>> 4. How long are direct upgrades supported for. During the time-based > >>>> releases discussion, we agreed to support direct upgrades for 2 years. > >> As > >>>> it happens, direct upgrades from 0.8.2 to 1.0.0 would not be supported > >> if > >>>> we follow this strictly. Not clear if we want to do that. > >>>> > >>>> 5. How long are older clients supported for. Current brokers support > >>>> clients all the way to 0.8.x. > >>>> > >>>> 6. How long are older brokers supported for. Current clients support > >>> 0.10.x > >>>> and newer brokers. > >>>> > >>>> 7. How long are message formats supported for. We never discussed > >> this. I > >>>> think 5 years would probably be the minimum. > >>>> > >>>> Ismael > >>>> > >>>> P.S. I'll file a JIRA to capture this information. > >>>> > >>>> On Wed, Jul 19, 2017 at 10:12 AM, Ismael Juma <ism...@juma.me.uk> > >> wrote: > >>>> > >>>>> Hi Stevo, > >>>>> > >>>>> Thanks for your feedback. We should definitely do a better job of > >>>>> documenting things. We basically follow semantic versioning, but it's > >>>>> currently a bit confusing because: > >>>>> > >>>>> 1. There are 4 segments in the version. The "0." part should be > >> ignored > >>>>> when deciding what is major, minor and patch at the moment, but many > >>>> people > >>>>> don't know this. Once we move to 1.0.0, that problem goes away. > >>>>> > >>>>> 2. To know what is a public API, you must check the Javadoc ( > >>>>> https://kafka.apache.org/0110/javadoc/index.html?org/ > >>>>> apache/kafka/clients/consumer/KafkaConsumer.html). If it's not > >> listed > >>>>> there, it's not public API. Ideally, it would be obvious from the > >>> package > >>>>> name (i.e. there would be "internals" in the name), but we are not > >>> there > >>>>> yet. The exception are the old Scala APIs, but they have all been > >>>>> deprecated and they will be removed eventually (the old Scala > >> consumers > >>>>> won't be removed until the June 2018 release at the earliest in order > >>> to > >>>>> give people time to migrate). > >>>>> > >>>>> 3. Even though we are following reasonably common practices, we > >> haven't > >>>>> documented them all in one place. It would be great to do it during > >> the > >>>>> next release cycle. > >>>>> > >>>>> A few comments below. > >>>>> > >>>>> On Wed, Jul 19, 2017 at 1:31 AM, Stevo Slavić <ssla...@gmail.com> > >>> wrote: > >>>>> > >>>>>> - APIs not labeled or labeled as stable > >>>>>> -- change in major version is only one that can break backward > >>>>>> compatibility (client APIs or behavior) > >>>>>> > >>>>> > >>>>> To clarify, stable APIs should not be changed in an incompatible way > >>>>> without a deprecation cycle. Independently of whether it's a major > >>>> release > >>>>> or not. > >>>>> > >>>>> > >>>>>> -- change in minor version can introduce new features, but not break > >>>>>> backward compatibility > >>>>>> -- change in patch version, is for bug fixes only. > >>>>>> > >>>>> > >>>>> Right, this has been the case for a while already. Also see > >> annotations > >>>>> below. > >>>>> > >>>>> > >>>>>> - APIs labeled as evolving can be broken in backward incompatible > >> way > >>> in > >>>>>> any release, but are assumed less likely to be broken compared to > >>>> unstable > >>>>>> APIs > >>>>>> - APIs labeled as unstable can be broken in backward incompatible > >> way > >>> in > >>>>>> any release, major, minor or patch > >>>>>> > >>>>> > >>>>> The relevant annotations do explain this: > >>>>> > >>>>> https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > >>>> common/annotation/ > >>>>> InterfaceStability.html > >>>>> https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > >>>> common/annotation/ > >>>>> InterfaceStability.Stable.html > >>>>> https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > >>>> common/annotation/ > >>>>> InterfaceStability.Evolving.html > >>>>> https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > >>>> common/annotation/ > >>>>> InterfaceStability.Unstable.html > >>>>> > >>>>> But we should have a section in our documentation as well. > >>>>> > >>>>> > >>>>>> - deprecated stable APIs are treated as any stable APIs, they can be > >>>>>> removed only in major release, are not allowed to be changed in > >>> backward > >>>>>> incompatible way in either patch or minor version release > >>>>>> > >>>>> > >>>>> Right, but note that stable non-deprecated APIs provide stronger > >>>>> guarantees in major releases (they can't be changed in an > >> incompatible > >>>> way). > >>>>> > >>>>>> > >>>>>> This means one should be able to upgrade server and recompile/deploy > >>>> apps > >>>>>> with clients to new minor.patch release with dependency version > >> change > >>>>>> being only change needed and there would be no drama. > >>>>>> > >>>>> > >>>>> That should have been the case for a while as long as you are using > >>>> stable > >>>>> public APIs. > >>>>> > >>>>>> > >>>>>> Practice/"features" like protocol version being a parameter, and > >>>>>> defaulting > >>>>>> to latest so auto updated with dependency update which introduces > >> new > >>>>>> protocol/behavior should not be used in public client APIs. To > >> switch > >>>>>> between backward incompatible APIs (contract and behaviors), ideally > >>>> user > >>>>>> should explicitly have to change code and not dependency only, but > >> at > >>>>>> least > >>>>>> it should be clearly communicated that there are breaking changes to > >>>>>> expect > >>>>>> even with just dependency update by e.g. giving major version > >> release > >>>>>> clear > >>>>>> meaning. If app dependency on Kafka client library minor.patch on > >> same > >>>>>> major is updated, and if there's a change in behavior or API > >> requiring > >>>> app > >>>>>> code change - it's a bug. > >>>>>> > >>>>> > >>>>> Hmm, if the protocol bump provides improved behaviour, that is not a > >>>>> backwards incompatible change though. So, I don't think I agree with > >>>> this. > >>>>> Of course, > >>>>> it does mean that _downgrading_ may cause loss of functionality. > >> That's > >>>>> OK, in my opinion. > >>>>> > >>>>> Change introduced contrary to the SLO, is OK to be reported as bug. > >>>>>> Everything else is improvement or feature request. > >>>>>> > >>>>>> If this was the case, and 1.0.0 was released today with APIs as they > >>> are > >>>>>> now, Scala client APIs even though deprecated would not break and > >>>> require > >>>>>> refactoring with every 1.* minor/patch release, and would only be > >>>> allowed > >>>>>> to be broken or removed in future major release, like 2.0.0 > >>>>>> > >>>>> > >>>>> Yes, that is the plan for any _public_ Scala client APIs that are > >> still > >>>>> present in 1.0.0. The public Scala client APIs are the producer and > >>>>> consumer, basically. Again, we should make this clear in our > >>>> documentation. > >>>>> Note that we have made an effort to keep those APIs compatible for > >>> quite > >>>> a > >>>>> while. It sounds like you have had some issues, were they related to > >>>> usage > >>>>> of internal Admin APIs by any chance (since we didn't have a public > >>>>> AdminClient API until very recently)? > >>>>> > >>>>>> > >>>>>> It should be also clear how long is each version supported - e.g. if > >>>>>> minor.patch had meaning that there are no backward incompatible > >>> changes, > >>>>>> it's OK to file a bug only for current major.minor.patch; previous > >>> major > >>>>>> and its last minor.patch can only have patches released up to some > >>> time > >>>>>> like 1 up to 3 months. > >>>>>> > >>>>> > >>>>> I am not sure I understood this point correctly. Can you please > >>> clarify? > >>>>> > >>>>> If there are changes in release cadence with new versioning, it > >> should > >>> be > >>>>>> clear too. > >>>>>> > >>>>> > >>>>> No changes are planned. We have started time-based releases less > >> than a > >>>>> year ago and they seem to be going well. > >>>>> > >>>>> Ismael > >>>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> -- Guozhang > >>> > >> > > > > > > > >
