Hi Yanquan, the current state of the 3.4.0 release is that it's still pending on the lineage PR [1] which I expect to be merged next week (the author is on vacation). The release cut would then happen right afterwards.
After the release cut, we can then bump to 4.0.0-SNAPSHOT and Flink 2.0-preview. @Qingsheng Ren <renqs...@gmail.com> and Leonard wanted to drive that release. I already prepared a bit by thoroughly annotating everything with Deprecated but the whole test side needs a bigger cleanup. It's probably also a good time to bump other dependencies. Could you please sync with the two release managers? At least Qingsheng is responsive in the Flink slack - I talked to him quite a bit there. If there is a pressing need to start earlier, we can also cut the 3.4 branch (which is then effectively the 3.3 branch) earlier and backport the lineage PR (it's just one commit ultimately). I'd leave that decision to the two release managers for 4.0.0 mentioned before. One thing to note for 4.0.0 is that we need to solve the transaction management issues with the KafkaSink [2]. It's blocking larger users from adopting the KafkaSink which will be the only option for Flink 2.0. I have started designing a solution. Best, Arvid [1] https://github.com/apache/flink-connector-kafka/pull/130 [2] https://issues.apache.org/jira/browse/FLINK-34554 On Tue, Nov 5, 2024 at 4:48 AM Yanquan Lv <decq12y...@gmail.com> wrote: > Hi, Arvid. > > It has been a month and we are glad to see that we have completed the > release of Kafka 3.3.0 targeting 1.19 and 1.20. > > Considering that Flink 2.0-preview1 has already been released, I would > like to know about our plans and progress for bumping to 2.0-preview1. > I tested the changes required for bump to 2.0-preview1 locally and found > that the adaptation changes made in the production code based on > FlinkKafkaProducer Depreciated work were relatively clear and the amount of > change was not significant. However, the headache was that there were many > adjustments needed in the test code. > > I would like to know if there is already work in the community to bump to > 2.0-preview1. If not, I can help complete this task (but some suggestions > may be needed for testing the adaptation in the code). > > > > > > > 2024年9月27日 16:23，Arvid Heise <ar...@apache.org> 写道： > > > > Dear Flink devs, > > > > I'd like to initiate three(!) Kafka connector releases. The main reason > for > > having three releases is that we have been slacking a bit in keeping up > > with the latest changes. > > > > Here is the summary: > > 1. Release kafka-3.3.0 targeting 1.19 and 1.20 (asap) > > - Incorporates lots of deprecations for Flink 2 including everything that > > is related to FlinkKafkaProducer (SinkFunction, FlinkKafkaConsumer > > (SourceFunction), and KafkaShuffle > > - Lots of bugfixes that are very relevant for 1.19 users (and probably > work > > with older releases) > > > > 2. Release kafka-3.4.0 targeting 1.20 (~1-2 weeks later) > > - Incorporates lineage tracing which is only available in 1.20 [1] (FLIP > > incorrectly says that it's avail in 1.19) > > - We have discussed some alternatives to this release in [2] but > basically > > having a separate release is the cleanest solution. > > - I'd like to linearize the releases to avoid having to do back or even > > forward ports > > > > 3. Release kafka-4.0.0 targeting 2.0-preview (~1-2 weeks later) > > - Much requested to get the connector out asap for the preview. (I think > > the old jar using the removed interfaces should still work) > > - Remove all deprecated things > > - General spring cleaning (trying to get rid of arch unit violations, > > migrate to JUnit5) > > - Should we relocate the TableAPI stuff to o.a.f.connectors? > > > > I'd appreciate any feedback and volunteers for RM ;) If you have pending > > PRs that should be part of any of those releases, please also write them. > > > > Best, > > > > Arvid > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-314%3A+Support+Customized+Job+Lineage+Listener > > [1] > > > https://github.com/apache/flink-connector-kafka/pull/111#issuecomment-2306382878 > >
