Thanks for the analysis Till. 1/ I think breaking binary compatibility is acceptable between minor releases (1.10 -> 1.11) as you suggested since the API is marked as @PublicEvolving.
2/ I'm quite torn about how to proceed with the 1.10.x patch release, but slightly leaning towards the "pragmatic" solution of keeping the change and adding a big warning to the 1.10.1 release announcement*. What do others think? If we do want to revert the change and follow up with a 1.10.2 release we should do it _before_ we announce 1.10.1 publicly. Doing it after 1.10.1 has been announced would only cause more friction for users. – Ufuk *I hope we don't lose user trust with this. I really would like users to be able to upgrade to the latest patch release without hesitation. On Wed, May 13, 2020 at 11:45 AM Till Rohrmann <trohrm...@apache.org> wrote: > Thanks for reporting this issue Seth. This is indeed a big problem. > > I looked into the problem and it seems we have the following situation: > > # 1.9.x --> 1.10.0 > > There is an API breaking change between 1.9.x and 1.10 due FLINK-13864 > because it introduced another generic parameter. I expected only few people > will be affected by this because one would have to store the builder in a > variable. > > 1.9.x is binary compatible with 1.10.0. > > # 1.10.0 -> 1.10.1 > > There is no API breaking change between 1.10.0 and 1.10.1. > > I changed the return type of the StreamingFileSink.forRowFormat and > .forBulkFormat to a subtype of StreamingFileSink.BulkFormatBuilder in > FLINK-16684. This causes the java.lang.NoSuchMethodError and the binary > incompatibility between 1.10.0 and 1.10.1. This is should not happen for > bug fix releases. > > W/o FLINK-16684, the StreamingFileSink builders cannot be used with Flink's > Scala API. > > # Options > > I think we should keep the fix for 1.11.0 and add a release note that we > are violating binary compatibility between 1.10 and 1.11. > > Now the question is what to do with the 1.10.x branch: > > a) We can revert the change to re-establish binary compatibility between > 1.9.x, 1.10.0 and 1.10.2 but not between 1.10.1 and 1.10.2. This would also > imply that we cannot use the StreamingFileSink builders with the Scala API. > > b) Keep the change and add a release note that our users need to recompile > their jobs. This would allow Flink 1.10.x with x >= 1 users to use > StreamingFileSink builders with the Scala API. > > I am not entirely sure which need weighs more: binary compatibility between > bug fix releases or a working API (small subset of it). Another aspect to > consider is how many people will migrate from 1.10.0 to 1.10.1 compared to > 1.y to 1.10.1 with y <= 9. If the former is very small then one might make > a case for keeping the change. > > What do the others think? > > https://issues.apache.org/jira/browse/FLINK-13864 > https://issues.apache.org/jira/browse/FLINK-16684 > > Cheers, > Till > > On Tue, May 12, 2020 at 7:26 PM Thomas Weise <t...@apache.org> wrote: > > > We also noticed that and had to make an adjustment downstream. > > > > It would be good to mention this in the release notes (if that's not > > already the case). > > > > Thomas > > > > > > On Tue, May 12, 2020 at 10:06 AM Seth Wiesman <sjwies...@gmail.com> > wrote: > > > > > Hi Everyone, > > > > > > I realize I'm about 7 hours late but I just realized there is a > breaking > > > API change in 1.10.1. FLINK-16684 changes the API of the streaming file > > > sink in a binary incompatible way. Since the release has been approved > > I'm > > > not sure the correct course of action but I wanted to bring this to the > > > communities attention. > > > > > > Seth > > > > > > https://issues.apache.org/jira/browse/FLINK-16684 > > > > > >
