Thanks for reporting this issue Seth. This is indeed a big problem. I looked into the problem and it seems we have the following situation:
# 1.9.x --> 1.10.0 There is an API breaking change between 1.9.x and 1.10 due FLINK-13864 because it introduced another generic parameter. I expected only few people will be affected by this because one would have to store the builder in a variable. 1.9.x is binary compatible with 1.10.0. # 1.10.0 -> 1.10.1 There is no API breaking change between 1.10.0 and 1.10.1. I changed the return type of the StreamingFileSink.forRowFormat and .forBulkFormat to a subtype of StreamingFileSink.BulkFormatBuilder in FLINK-16684. This causes the java.lang.NoSuchMethodError and the binary incompatibility between 1.10.0 and 1.10.1. This is should not happen for bug fix releases. W/o FLINK-16684, the StreamingFileSink builders cannot be used with Flink's Scala API. # Options I think we should keep the fix for 1.11.0 and add a release note that we are violating binary compatibility between 1.10 and 1.11. Now the question is what to do with the 1.10.x branch: a) We can revert the change to re-establish binary compatibility between 1.9.x, 1.10.0 and 1.10.2 but not between 1.10.1 and 1.10.2. This would also imply that we cannot use the StreamingFileSink builders with the Scala API. b) Keep the change and add a release note that our users need to recompile their jobs. This would allow Flink 1.10.x with x >= 1 users to use StreamingFileSink builders with the Scala API. I am not entirely sure which need weighs more: binary compatibility between bug fix releases or a working API (small subset of it). Another aspect to consider is how many people will migrate from 1.10.0 to 1.10.1 compared to 1.y to 1.10.1 with y <= 9. If the former is very small then one might make a case for keeping the change. What do the others think? https://issues.apache.org/jira/browse/FLINK-13864 https://issues.apache.org/jira/browse/FLINK-16684 Cheers, Till On Tue, May 12, 2020 at 7:26 PM Thomas Weise <t...@apache.org> wrote: > We also noticed that and had to make an adjustment downstream. > > It would be good to mention this in the release notes (if that's not > already the case). > > Thomas > > > On Tue, May 12, 2020 at 10:06 AM Seth Wiesman <sjwies...@gmail.com> wrote: > > > Hi Everyone, > > > > I realize I'm about 7 hours late but I just realized there is a breaking > > API change in 1.10.1. FLINK-16684 changes the API of the streaming file > > sink in a binary incompatible way. Since the release has been approved > I'm > > not sure the correct course of action but I wanted to bring this to the > > communities attention. > > > > Seth > > > > https://issues.apache.org/jira/browse/FLINK-16684 > > >
