Thanks for the comment, Till and Thomas. As far as I know there are some users who have just upgraded their Flink version from 1.8 / 1.9 to Flink 1.12 and might not upgrade the version in 6 months or more. There are also some organizations that have the strategy of not running the latest version of a project, but only the second latest version with bug fixes. So those users may still benefit from the backport. However, arguably the old Kafka source is there anyways in 1.12, so they should not be blocked on having the new source.
I am leaning towards backporting the fixes mainly because this way we might have more users migrating to the new Source and provide feedback. It will take some time for the users to pick up 1.13, especially for the users running Flink at large scale. So backporting the fixes to 1.12 would help get the new source to be used sooner. Thanks, Jiangjie (Becket) Qin On Thu, Apr 8, 2021 at 12:40 AM Thomas Weise <t...@apache.org> wrote: > Hi, > > Thanks for fixing the new KafkaSource issues. > > I'm interested in using these fixes with 1.12 for experimental purposes. > > +1 for backporting. 1.12 is the current stable release and users who would > like to try the FLIP-27 sources are likely to use that release. > > Thomas > > On Wed, Apr 7, 2021 at 2:50 AM Till Rohrmann <trohrm...@apache.org> wrote: > > > Hi Becket, > > > > If I remember correctly, then we deliberately not documented the Kafka > > connector in the 1.12 release. Hence, from this point there should be no > > need to backport any fixes because users are not aware of this feature. > > > > On the other hand this also means that we should be able to break > anything > > we want to. Consequently, backporting these fixes should be possible. > > > > The question would probably be whether we want to ship new features with > a > > bug fix release. Do we know of any users who want to use the new Kafka > > source, are using the 1.12 version and cannot upgrade to 1.13 once it is > > released? If this is the case, then this could be an argument for > shipping > > this feature with a bug fix release. If not, then we could save some work > > by not backporting it. > > > > Cheers, > > Till > > > > On Wed, Apr 7, 2021 at 10:43 AM Becket Qin <becket....@gmail.com> wrote: > > > > > Hi folks, > > > > > > I'd like to start a discussion thread about backporting some FLIP-27 > > Kafka > > > source connector fixes to release-1.12. These fixes include some API > > > changes and thus needs a public discussion. > > > > > > The tickets in question are following: > > > https://issues.apache.org/jira/browse/FLINK-20379 > > > https://issues.apache.org/jira/browse/FLINK-20114 > > > https://issues.apache.org/jira/browse/FLINK-21817 > > > > > > Without these fixes, the FLIP-27 Kafka source in release-1.12 is not > > really > > > usable, and the API changes only affect the Kafka Source. So it seems > > > breaking the API in this case is still worthwhile. > > > > > > It would be good to see what others think. > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > > > >
