+1 for the release note addition Historically Flink X.Y.0 releases have been prone to issues like this. And I suspect that many users would prefer to upgrade to X.Y.1 due to other bugs that need to be shaken out.
For the future, it would be great to eliminate the possibility of incompatible changes to user-facing classes for patch releases though. Running japicmp as part of CI might be a good option. Thanks, Thomas On Wed, May 13, 2020 at 7:54 AM Seth Wiesman <sjwies...@gmail.com> wrote: > +1 Ufuk's pragmatic solution to update the release notes with a public > notice, I have commented on the release notes PR. > > https://github.com/apache/flink-web/pull/330 > > Seth > > On Wed, May 13, 2020 at 8:30 AM Chesnay Schepler <ches...@apache.org> > wrote: > > > IIRC @PublicEvolving had no /real/ guarantees, from the 1.0.0 > announcement: > > > > /"Flink 1.0.0 introduces interface stability annotations for API classes > > and methods. Interfaces defined as //|@Public|//are guaranteed to remain > > stable across all releases of the 1.x series. The > > //|@PublicEvolving|//annotation marks API features that may be subject > > to change in future versions."/ > > > > 1) We don't have a process for promoting things to @Public because we > > have actually never done that. Based on our current rules, it is an > > addition to the public API (technically even more so than most FLIPs), > > and hence should go through a FLIP and vote. > > Ideally we evaluate existing @PublicEvolving API's for promotion on each > > release. > > > > 2) Yes, I think this is an excellent idea; as Ufuk said users should not > > be worried about upgrading to the next minor version. Not sure how easy > > this is to implement; but it basically means either modifying or adding > > new new japicmp execution when forking the release-X.Y branches. > > > > On 13/05/2020 15:19, Till Rohrmann wrote: > > > A small addendum: The project currently enforces only binary > > compatibility > > > for classes which are annotated with @Public. The StreamingFileSink is > > > annotated with @PublicEvolving. > > > > > > I am not sure whether the community properly defined what kind of > > > stability/compatibility guarantees we provide for @PublicEvolving > > classes. > > > > > > We can derive two follow-up discussions from this: > > > > > > 1) Should the StreamingFileSink be made @Public? Should > > > other @PublicEvolving classes be promoted? What is the process here? > > > > > > 2) Should we enforce binary compatibility of @PublicEvolving classes > > > between bug fix releases and allow breaking changes between minor/major > > > releases? > > > > > > Cheers, > > > Till > > > > > > On Wed, May 13, 2020 at 2:30 PM Ufuk Celebi <u...@apache.org> wrote: > > > > > >> Thanks for the analysis Till. > > >> > > >> 1/ I think breaking binary compatibility is acceptable between minor > > >> releases (1.10 -> 1.11) as you suggested since the API is marked > > >> as @PublicEvolving. > > >> > > >> 2/ I'm quite torn about how to proceed with the 1.10.x patch release, > > but > > >> slightly leaning towards the "pragmatic" solution of keeping the > change > > and > > >> adding a big warning to the 1.10.1 release announcement*. What do > others > > >> think? If we do want to revert the change and follow up with a 1.10.2 > > >> release we should do it _before_ we announce 1.10.1 publicly. Doing it > > >> after 1.10.1 has been announced would only cause more friction for > > users. > > >> > > >> – Ufuk > > >> > > >> *I hope we don't lose user trust with this. I really would like users > > to be > > >> able to upgrade to the latest patch release without hesitation. > > >> > > >> On Wed, May 13, 2020 at 11:45 AM Till Rohrmann <trohrm...@apache.org> > > >> wrote: > > >> > > >>> Thanks for reporting this issue Seth. This is indeed a big problem. > > >>> > > >>> I looked into the problem and it seems we have the following > situation: > > >>> > > >>> # 1.9.x --> 1.10.0 > > >>> > > >>> There is an API breaking change between 1.9.x and 1.10 due > FLINK-13864 > > >>> because it introduced another generic parameter. I expected only few > > >> people > > >>> will be affected by this because one would have to store the builder > > in a > > >>> variable. > > >>> > > >>> 1.9.x is binary compatible with 1.10.0. > > >>> > > >>> # 1.10.0 -> 1.10.1 > > >>> > > >>> There is no API breaking change between 1.10.0 and 1.10.1. > > >>> > > >>> I changed the return type of the StreamingFileSink.forRowFormat and > > >>> .forBulkFormat to a subtype of StreamingFileSink.BulkFormatBuilder in > > >>> FLINK-16684. This causes the java.lang.NoSuchMethodError and the > binary > > >>> incompatibility between 1.10.0 and 1.10.1. This is should not happen > > for > > >>> bug fix releases. > > >>> > > >>> W/o FLINK-16684, the StreamingFileSink builders cannot be used with > > >> Flink's > > >>> Scala API. > > >>> > > >>> # Options > > >>> > > >>> I think we should keep the fix for 1.11.0 and add a release note that > > we > > >>> are violating binary compatibility between 1.10 and 1.11. > > >>> > > >>> Now the question is what to do with the 1.10.x branch: > > >>> > > >>> a) We can revert the change to re-establish binary compatibility > > between > > >>> 1.9.x, 1.10.0 and 1.10.2 but not between 1.10.1 and 1.10.2. This > would > > >> also > > >>> imply that we cannot use the StreamingFileSink builders with the > Scala > > >> API. > > >>> b) Keep the change and add a release note that our users need to > > >> recompile > > >>> their jobs. This would allow Flink 1.10.x with x >= 1 users to use > > >>> StreamingFileSink builders with the Scala API. > > >>> > > >>> I am not entirely sure which need weighs more: binary compatibility > > >> between > > >>> bug fix releases or a working API (small subset of it). Another > aspect > > to > > >>> consider is how many people will migrate from 1.10.0 to 1.10.1 > compared > > >> to > > >>> 1.y to 1.10.1 with y <= 9. If the former is very small then one might > > >> make > > >>> a case for keeping the change. > > >>> > > >>> What do the others think? > > >>> > > >>> https://issues.apache.org/jira/browse/FLINK-13864 > > >>> https://issues.apache.org/jira/browse/FLINK-16684 > > >>> > > >>> Cheers, > > >>> Till > > >>> > > >>> On Tue, May 12, 2020 at 7:26 PM Thomas Weise <t...@apache.org> wrote: > > >>> > > >>>> We also noticed that and had to make an adjustment downstream. > > >>>> > > >>>> It would be good to mention this in the release notes (if that's not > > >>>> already the case). > > >>>> > > >>>> Thomas > > >>>> > > >>>> > > >>>> On Tue, May 12, 2020 at 10:06 AM Seth Wiesman <sjwies...@gmail.com> > > >>> wrote: > > >>>>> Hi Everyone, > > >>>>> > > >>>>> I realize I'm about 7 hours late but I just realized there is a > > >>> breaking > > >>>>> API change in 1.10.1. FLINK-16684 changes the API of the streaming > > >> file > > >>>>> sink in a binary incompatible way. Since the release has been > > >> approved > > >>>> I'm > > >>>>> not sure the correct course of action but I wanted to bring this to > > >> the > > >>>>> communities attention. > > >>>>> > > >>>>> Seth > > >>>>> > > >>>>> https://issues.apache.org/jira/browse/FLINK-16684 > > >>>>> > > > > >
