The vote thread can be found here https://lists.apache.org/thread.html/rc58099fb0e31d0eac951a7bbf7f8bda8b7b65c9ed0c04622f5333745%40%3Cdev.flink.apache.org%3E .
Cheers, Till On Fri, May 15, 2020 at 3:03 PM Till Rohrmann <trohrm...@apache.org> wrote: > I completely agree that there are many other aspect of our guarantees and > processes around the @Public and @PublicEvolving classes which need to be > discussed and properly defined. For the sake of keeping this discussion > thread narrowly scoped, I would suggest to start a separate discussion > about the following points (not exhaustive): > > - What should be annotated with @Public and @PublicEvolving? > - Process for transforming @PublicEvolving into @Public; How to ensure > that @PublicEvolving will eventually be promoted to @Public? > - Process of retiring a @Public/@PublicEvolving API > > I will start a vote thread about the change I proposed here which is to > ensure API and binary compatibility for @PublicEvolving classes between > bugfix releases (x.y.z and x.y.u). > > Cheers, > Till > > On Fri, May 15, 2020 at 6:33 AM Zhu Zhu <reed...@gmail.com> wrote: > >> +1 for "API + binary compatibility for @PublicEvolving classes for all bug >> fix >> releases in a minor release (x.y.z is compatible to x.y.u)" >> >> This @PublicEnvolving would then be a hard limit to changes. >> So it's important to rethink the policy towards using it, as Stephan >> proposed. >> >> I think any Flink interfaces that are visible to users should be >> explicitly >> marked as @Public or @PublicEnvolving. >> Any other interfaces should not be marked as @Public/@PublicEnvolving. >> This would be essential for us to check whether we are breaking any user >> faced interfaces unexpectedly. >> The only exception would be the case that we had to expose a method/class >> due to implementation limitations, it should be explicitly marked it >> as @Internal. >> >> Thanks, >> Zhu Zhu >> >> Yun Tang <myas...@live.com> 于2020年5月15日周五 上午11:41写道： >> >> > +1 for this idea, and I also like Xintong's suggestion to make it >> > explicitly when the @PublicEvolving API could upgrade to @Public API. >> > If we have the rule to upgrade API stable level but not define the clear >> > timeline, I'm afraid not everyone have the enthusiasm to upgrade this. >> > >> > The minor suggestion is that I think two major release (which is x.y.0 >> as >> > Chesnay clarified) might be a bit quick. From the release history [1], >> > Flink bump major version every 3 ~ 6 months and two major release gap >> > could only be at least half a year. >> > I think half a year might be a bit too frequent for users to collect >> > enough feedbacks, and upgrading API stable level every 3 major versions >> > should be better. >> > >> > [1] https://flink.apache.org/downloads.html#flink >> > >> > Best >> > Yun Tang >> > >> > >> > ________________________________ >> > From: Xintong Song <tonysong...@gmail.com> >> > Sent: Friday, May 15, 2020 11:04 >> > To: dev <dev@flink.apache.org> >> > Subject: Re: [DISCUSS] Stability guarantees for @PublicEvolving classes >> > >> > ### Documentation on API compatibility policies >> > >> > Do we have any formal documentation about the API compatibility >> policies? >> > The only things I found are: >> > >> > - In the release announcement (take 1.10.0 as an example) [1]: >> > "This version is API-compatible with previous 1.x releases for APIs >> > annotated with the @Public annotation." >> > - JavaDoc for Public [2] and PublicEvolving [3]. >> > >> > I think we might have a formal documentation, clearly state our policies >> > for API compatibility. >> > >> > - What does the annotations mean >> > - In what circumstance would the APIs remain compatible / become >> > incompatible >> > - How do APIs retire (e.g., first deprecated then removed?) >> > >> > Maybe there is already such kind of documentation that I overlooked? If >> so, >> > we probably want to make it more explicit and easy-to-find. >> > >> > ### @Public vs. @PublicEvolving for new things >> > >> > I share Stephan's concern that, with @PublicEvolving used for every new >> > feature and rarely upgraded to @Public, we are practically making no >> > compatibility guarantee between minor versions (x.y.* / x.z.*). On the >> > other hand, I think in many circumstances we do need some time to >> collect >> > feedbacks for new features before we have enough confidence to make the >> > commitment that our APIs are stable. Therefore, it makes more sense to >> me >> > to first make new features @PublicEvolving and then upgrade to @Public >> in >> > the next one or two releases (unless there's a good reason to further >> > postpone it). >> > >> > I think the key point is how do we make sure the @PublicEvolving >> features >> > upgrade to @Public. Maybe we can add a parameter to indicate the >> expected >> > upgrading version. E.g., a new feature introduced in release 1.10.0 >> might >> > be annotated as @PublicEvolving("1.12.0"), indicating that it is >> expected >> > to be upgraded to @Public in release 1.12.0. We can check the >> annotations >> > against the version automatically, forcing to either upgrad the feature >> > to @Public or explicitly postpone it by modifying the annotation >> parameter >> > (if there's a good reason). >> > >> > Additionally, we can do the similar for deprecated features / APIs, >> > reminding us to remove things annotated as @Deprecated at certain time. >> > >> > Thank you~ >> > >> > Xintong Song >> > >> > >> > [1] https://flink.apache.org/news/2020/02/11/release-1.10.0.html >> > >> > [2] >> > >> > >> https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/annotation/Public.html >> > >> > [3] >> > >> > >> https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/annotation/PublicEvolving.html >> > >> > >> > >> > >> > On Thu, May 14, 2020 at 8:22 PM Stephan Ewen <se...@apache.org> wrote: >> > >> > > I just want to throw in that we also need to rethink our policy >> towards >> > > using @PublicEvolving. >> > > >> > > We often introduce this easily (for every new feature) and rarely >> (almost >> > > never) upgrade it to @Public. This kind of leads the idea behind >> stable >> > API >> > > guarantees ad absurdum. >> > > >> > > I would suggest that we make @PublicEvolving an exception that needs a >> > good >> > > reason rather than for everything that is new (when we don't want to >> be >> > > bothered with thinking about compatibility). >> > > >> > > >> > > On Thu, May 14, 2020 at 1:05 PM Xintong Song <tonysong...@gmail.com> >> > > wrote: >> > > >> > > > Thanks for the clarification. >> > > > +1 for keeping the current guarantees for @Public. >> > > > >> > > > Thank you~ >> > > > >> > > > Xintong Song >> > > > >> > > > >> > > > >> > > > On Thu, May 14, 2020 at 6:07 PM Till Rohrmann <trohrm...@apache.org >> > >> > > > wrote: >> > > > >> > > > > Sorry for the confusion. @Public classes are guaranteed to be >> stable >> > > > > between releases x.y.z and x.u.v (minor and bug fix release; >> naming >> > is >> > > > > indeed a bit off here) and we can break it with major releases >> (x.0.0 >> > > and >> > > > > y.0.0). >> > > > > >> > > > > @Tison I would like to make what to include in the public API, >> hence >> > > what >> > > > > to annotate with @Public and @PublicEvolving, a separate >> discussion. >> > > > > >> > > > > Cheers, >> > > > > Till >> > > > > >> > > > > On Thu, May 14, 2020 at 11:48 AM tison <wander4...@gmail.com> >> wrote: >> > > > > >> > > > >> Thanks for starting this discussion! >> > > > >> >> > > > >> I agree turn on japicmp on PublicEvolving among bugfix releases >> is a >> > > nit >> > > > >> win. >> > > > >> >> > > > >> @Xintong Song <tonysong...@gmail.com> I think @Public guarantee >> is >> > > good >> > > > >> enough, the problem is a reachable 2.0 plan. >> > > > >> >> > > > >> My concern is more on classes that have no annotation but our >> > > developers >> > > > >> regard as "something that should be stable". Previously I was >> > required >> > > > to >> > > > >> keep compatibility of ClusterClient & HighAvailabilityServices >> > because >> > > > >> they >> > > > >> might be depended on by user. >> > > > >> >> > > > >> Best, >> > > > >> tison. >> > > > >> >> > > > >> >> > > > >> Dawid Wysakowicz <dwysakow...@apache.org> 于2020年5月14日周四 >> 下午5:08写道： >> > > > >> >> > > > >> > I also like the proposal for keeping the binary compatibility >> of >> > > > >> > @PublicEvolving for bugfix releases. >> > > > >> > >> > > > >> > As for the @Public classes I think the current guarantees are >> good >> > > > >> enough. >> > > > >> > >> > > > >> > Best, >> > > > >> > >> > > > >> > Dawid >> > > > >> > >> > > > >> > On 14/05/2020 10:49, Jingsong Li wrote: >> > > > >> > > Thanks Till for starting this discussion. >> > > > >> > > >> > > > >> > > +1 for enabling the japicmp-maven-plugin for @PublicEvolving >> for >> > > bug >> > > > >> fix >> > > > >> > > releases. >> > > > >> > > Bug fix should just be user imperceptible bug fix. Should not >> > > affect >> > > > >> API >> > > > >> > > and binary compatibility. >> > > > >> > > >> > > > >> > > And even PublicEvolving api change for "y" release, we should >> > > expose >> > > > >> it >> > > > >> > in >> > > > >> > > dev mail list for discussing or a FLIP? >> > > > >> > > >> > > > >> > > BTW, public api can be changed by major releases? In >> annotation >> > > > >> comments: >> > > > >> > > "Only major releases (1.0, 2.0, 3.0) can break interfaces >> with >> > > this >> > > > >> > > annotation". >> > > > >> > > >> > > > >> > > Best, >> > > > >> > > Jingsong Lee >> > > > >> > > >> > > > >> > > On Thu, May 14, 2020 at 4:30 PM Till Rohrmann < >> > > trohrm...@apache.org >> > > > > >> > > > >> > wrote: >> > > > >> > > >> > > > >> > >> Dear community, >> > > > >> > >> >> > > > >> > >> in the latest 1.10.1 bug fix release I introduced a binary >> > > > >> incompatible >> > > > >> > >> change to a class which is annotated with @PublicEvolving >> [1]. >> > > > While >> > > > >> > this >> > > > >> > >> change is technically ok since we only provide API and >> binary >> > > > >> > compatibility >> > > > >> > >> for @Public classes across releases, it raised the question >> > > whether >> > > > >> we >> > > > >> > >> can't do better. >> > > > >> > >> >> > > > >> > >> For our users it might be surprising and really annoying >> that >> > > they >> > > > >> > cannot >> > > > >> > >> simply upgrade to the latest bug fix release without >> > recompiling >> > > > the >> > > > >> > >> program or even having to change the source code of an >> > > > application. I >> > > > >> > >> believe we would provide a much better experience if we >> ensured >> > > > that >> > > > >> bug >> > > > >> > >> fix releases maintain API and binary compatibility also for >> > > > >> > @PublicEvolving >> > > > >> > >> classes. Hence my proposal would be to tighten the stability >> > > > >> guarantees >> > > > >> > the >> > > > >> > >> following way: >> > > > >> > >> >> > > > >> > >> * API + binary compatibility for @Public classes across all >> > > > releases >> > > > >> > (x.y.z >> > > > >> > >> is compatible to u.v.w) >> > > > >> > >> * API + binary compatibility for @PublicEvolving classes for >> > all >> > > > bug >> > > > >> fix >> > > > >> > >> releases in a minor release (x.y.z is compatible to x.y.u) >> > > > >> > >> >> > > > >> > >> This would entail that we can change @PublicEvolving classes >> > only >> > > > >> across >> > > > >> > >> minor/major releases. >> > > > >> > >> >> > > > >> > >> Practically this would mean that we enable the >> > > japicmp-maven-plugin >> > > > >> > >> for @PublicEvolving for bug fix releases. >> > > > >> > >> >> > > > >> > >> What do you think? >> > > > >> > >> >> > > > >> > >> [1] >> > > > >> > >> >> > > > >> > >> >> > > > >> > >> > > > >> >> > > > >> > > >> > >> https://lists.apache.org/thread.html/r293768d13d08149d756e0bf91be52372edb444c317535d1d5a496c3e%40%3Cdev.flink.apache.org%3E >> > > > >> > >> >> > > > >> > >> Cheers, >> > > > >> > >> Till >> > > > >> > >> >> > > > >> > > >> > > > >> > >> > > > >> > >> > > > >> >> > > > > >> > > > >> > > >> > >> >
