### Documentation on API compatibility policies
Do we have any formal documentation about the API compatibility policies?
The only things I found are:
- In the release announcement (take 1.10.0 as an example) [1]:
"This version is API-compatible with previous 1.x releases for APIs
annotated with the @Public annotation."
- JavaDoc for Public [2] and PublicEvolving [3].
I think we might have a formal documentation, clearly state our policies
for API compatibility.
- What does the annotations mean
- In what circumstance would the APIs remain compatible / become
incompatible
- How do APIs retire (e.g., first deprecated then removed?)
Maybe there is already such kind of documentation that I overlooked? If so,
we probably want to make it more explicit and easy-to-find.
### @Public vs. @PublicEvolving for new things
I share Stephan's concern that, with @PublicEvolving used for every new
feature and rarely upgraded to @Public, we are practically making no
compatibility guarantee between minor versions (x.y.* / x.z.*). On the
other hand, I think in many circumstances we do need some time to collect
feedbacks for new features before we have enough confidence to make the
commitment that our APIs are stable. Therefore, it makes more sense to me
to first make new features @PublicEvolving and then upgrade to @Public in
the next one or two releases (unless there's a good reason to further
postpone it).
I think the key point is how do we make sure the @PublicEvolving features
upgrade to @Public. Maybe we can add a parameter to indicate the expected
upgrading version. E.g., a new feature introduced in release 1.10.0 might
be annotated as @PublicEvolving("1.12.0"), indicating that it is expected
to be upgraded to @Public in release 1.12.0. We can check the annotations
against the version automatically, forcing to either upgrad the feature
to @Public or explicitly postpone it by modifying the annotation parameter
(if there's a good reason).
Additionally, we can do the similar for deprecated features / APIs,
reminding us to remove things annotated as @Deprecated at certain time.
Thank you~
Xintong Song
[1] https://flink.apache.org/news/2020/02/11/release-1.10.0.html
[2]
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/annotation/Public.html
[3]
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/annotation/PublicEvolving.html
On Thu, May 14, 2020 at 8:22 PM Stephan Ewen <se...@apache.org> wrote:
> I just want to throw in that we also need to rethink our policy towards
> using @PublicEvolving.
>
> We often introduce this easily (for every new feature) and rarely (almost
> never) upgrade it to @Public. This kind of leads the idea behind stable API
> guarantees ad absurdum.
>
> I would suggest that we make @PublicEvolving an exception that needs a good
> reason rather than for everything that is new (when we don't want to be
> bothered with thinking about compatibility).
>
>
> On Thu, May 14, 2020 at 1:05 PM Xintong Song <tonysong...@gmail.com>
> wrote:
>
> > Thanks for the clarification.
> > +1 for keeping the current guarantees for @Public.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Thu, May 14, 2020 at 6:07 PM Till Rohrmann <trohrm...@apache.org>
> > wrote:
> >
> > > Sorry for the confusion. @Public classes are guaranteed to be stable
> > > between releases x.y.z and x.u.v (minor and bug fix release; naming is
> > > indeed a bit off here) and we can break it with major releases (x.0.0
> and
> > > y.0.0).
> > >
> > > @Tison I would like to make what to include in the public API, hence
> what
> > > to annotate with @Public and @PublicEvolving, a separate discussion.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Thu, May 14, 2020 at 11:48 AM tison <wander4...@gmail.com> wrote:
> > >
> > >> Thanks for starting this discussion!
> > >>
> > >> I agree turn on japicmp on PublicEvolving among bugfix releases is a
> nit
> > >> win.
> > >>
> > >> @Xintong Song <tonysong...@gmail.com> I think @Public guarantee is
> good
> > >> enough, the problem is a reachable 2.0 plan.
> > >>
> > >> My concern is more on classes that have no annotation but our
> developers
> > >> regard as "something that should be stable". Previously I was required
> > to
> > >> keep compatibility of ClusterClient & HighAvailabilityServices because
> > >> they
> > >> might be depended on by user.
> > >>
> > >> Best,
> > >> tison.
> > >>
> > >>
> > >> Dawid Wysakowicz <dwysakow...@apache.org> 于2020年5月14日周四 下午5:08写道：
> > >>
> > >> > I also like the proposal for keeping the binary compatibility of
> > >> > @PublicEvolving for bugfix releases.
> > >> >
> > >> > As for the @Public classes I think the current guarantees are good
> > >> enough.
> > >> >
> > >> > Best,
> > >> >
> > >> > Dawid
> > >> >
> > >> > On 14/05/2020 10:49, Jingsong Li wrote:
> > >> > > Thanks Till for starting this discussion.
> > >> > >
> > >> > > +1 for enabling the japicmp-maven-plugin for @PublicEvolving for
> bug
> > >> fix
> > >> > > releases.
> > >> > > Bug fix should just be user imperceptible bug fix. Should not
> affect
> > >> API
> > >> > > and binary compatibility.
> > >> > >
> > >> > > And even PublicEvolving api change for "y" release, we should
> expose
> > >> it
> > >> > in
> > >> > > dev mail list for discussing or a FLIP?
> > >> > >
> > >> > > BTW, public api can be changed by major releases? In annotation
> > >> comments:
> > >> > > "Only major releases (1.0, 2.0, 3.0) can break interfaces with
> this
> > >> > > annotation".
> > >> > >
> > >> > > Best,
> > >> > > Jingsong Lee
> > >> > >
> > >> > > On Thu, May 14, 2020 at 4:30 PM Till Rohrmann <
> trohrm...@apache.org
> > >
> > >> > wrote:
> > >> > >
> > >> > >> Dear community,
> > >> > >>
> > >> > >> in the latest 1.10.1 bug fix release I introduced a binary
> > >> incompatible
> > >> > >> change to a class which is annotated with @PublicEvolving [1].
> > While
> > >> > this
> > >> > >> change is technically ok since we only provide API and binary
> > >> > compatibility
> > >> > >> for @Public classes across releases, it raised the question
> whether
> > >> we
> > >> > >> can't do better.
> > >> > >>
> > >> > >> For our users it might be surprising and really annoying that
> they
> > >> > cannot
> > >> > >> simply upgrade to the latest bug fix release without recompiling
> > the
> > >> > >> program or even having to change the source code of an
> > application. I
> > >> > >> believe we would provide a much better experience if we ensured
> > that
> > >> bug
> > >> > >> fix releases maintain API and binary compatibility also for
> > >> > @PublicEvolving
> > >> > >> classes. Hence my proposal would be to tighten the stability
> > >> guarantees
> > >> > the
> > >> > >> following way:
> > >> > >>
> > >> > >> * API + binary compatibility for @Public classes across all
> > releases
> > >> > (x.y.z
> > >> > >> is compatible to u.v.w)
> > >> > >> * API + binary compatibility for @PublicEvolving classes for all
> > bug
> > >> fix
> > >> > >> releases in a minor release (x.y.z is compatible to x.y.u)
> > >> > >>
> > >> > >> This would entail that we can change @PublicEvolving classes only
> > >> across
> > >> > >> minor/major releases.
> > >> > >>
> > >> > >> Practically this would mean that we enable the
> japicmp-maven-plugin
> > >> > >> for @PublicEvolving for bug fix releases.
> > >> > >>
> > >> > >> What do you think?
> > >> > >>
> > >> > >> [1]
> > >> > >>
> > >> > >>
> > >> >
> > >>
> >
> https://lists.apache.org/thread.html/r293768d13d08149d756e0bf91be52372edb444c317535d1d5a496c3e%40%3Cdev.flink.apache.org%3E
> > >> > >>
> > >> > >> Cheers,
> > >> > >> Till
> > >> > >>
> > >> > >
> > >> >
> > >> >
> > >>
> > >
> >
>
