Hi David, thank you very much for your reply.
> Some thoughts on whether we need the 3 deliverables. And whether we could follow more traditional fixpack numbering: > I see that there is already a release for 1.19 https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka/3.2.0-1.19 . I am wondering why we need the first deliverable. If we need it for the bug fixes , why not have a 3.2.1? I forgot the most important part of the first release: drop Flink 1.18 support and add Flink 1.20 support. Hence I wouldn't want to mix that into a bugfix release. I think this is in line with the previous minor releases. > I assume that kafka-3.4.0 will not work with previous Flink releases. Would it be worth have a config switch to enable the lineage in the connector so that we could use it with 1.19? We could maybe do a 3.3 if this was the case. Yes, as outlined in the discussion linked in the original message, we need to mixin new interfaces. Afaik classloading will fail if the interfaces are not present, even if the methods are not used. So I don't see how we can use a config switch to make it happen (except with code duplication). However, I'm grateful for any ideas to avoid this release. Best, Arvid On Fri, Sep 27, 2024 at 11:11 AM David Radley <david_rad...@uk.ibm.com> wrote: > Hi Arvid, > Some thoughts on whether we need the 3 deliverables. And whether we could > follow more traditional fixpack numbering: > I see that there is already a release for 1.19 > https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka/3.2.0-1.19 > . I am wondering why we need the first deliverable. If we need it for the > bug fixes , why not have a 3.2.1? > I assume that kafka-3.4.0 will not work with previous Flink releases. > Would it be worth have a config switch to enable the lineage in the > connector so that we could use it with 1.19? We could maybe do a 3.3 if > this was the case. > > WDYT? > Kind regards, David. > > > > From: Arvid Heise <ar...@apache.org> > Date: Friday, 27 September 2024 at 09:24 > To: dev@flink.apache.org <dev@flink.apache.org> > Subject: [EXTERNAL] Kafka connector releases > Dear Flink devs, > > I'd like to initiate three(!) Kafka connector releases. The main reason for > having three releases is that we have been slacking a bit in keeping up > with the latest changes. > > Here is the summary: > 1. Release kafka-3.3.0 targeting 1.19 and 1.20 (asap) > - Incorporates lots of deprecations for Flink 2 including everything that > is related to FlinkKafkaProducer (SinkFunction, FlinkKafkaConsumer > (SourceFunction), and KafkaShuffle > - Lots of bugfixes that are very relevant for 1.19 users (and probably work > with older releases) > > 2. Release kafka-3.4.0 targeting 1.20 (~1-2 weeks later) > - Incorporates lineage tracing which is only available in 1.20 [1] (FLIP > incorrectly says that it's avail in 1.19) > - We have discussed some alternatives to this release in [2] but basically > having a separate release is the cleanest solution. > - I'd like to linearize the releases to avoid having to do back or even > forward ports > > 3. Release kafka-4.0.0 targeting 2.0-preview (~1-2 weeks later) > - Much requested to get the connector out asap for the preview. (I think > the old jar using the removed interfaces should still work) > - Remove all deprecated things > - General spring cleaning (trying to get rid of arch unit violations, > migrate to JUnit5) > - Should we relocate the TableAPI stuff to o.a.f.connectors? > > I'd appreciate any feedback and volunteers for RM ;) If you have pending > PRs that should be part of any of those releases, please also write them. > > Best, > > Arvid > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-314%3A+Support+Customized+Job+Lineage+Listener > [1] > > https://github.com/apache/flink-connector-kafka/pull/111#issuecomment-2306382878 > > Unless otherwise stated above: > > IBM United Kingdom Limited > Registered in England and Wales with number 741598 > Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU >
