Thanks @Yanquan for track this thread, and thanks @Arvid for the information, it makes sense to me
Qingsheng will drive the release of flink-2.0-preview of Flink Kafka connector, and I’d like assist it too. Best, Leonard > 2024年11月6日 下午6:58，Arvid Heise <ahe...@confluent.io.INVALID> 写道： > > Hi Yanquan, > > the current state of the 3.4.0 release is that it's still pending on the > lineage PR [1] which I expect to be merged next week (the author is on > vacation). The release cut would then happen right afterwards. > > After the release cut, we can then bump to 4.0.0-SNAPSHOT and Flink > 2.0-preview. @Qingsheng Ren <renqs...@gmail.com> and Leonard wanted to > drive that release. I already prepared a bit by thoroughly annotating > everything with Deprecated but the whole test side needs a bigger cleanup. > It's probably also a good time to bump other dependencies. > Could you please sync with the two release managers? At least Qingsheng is > responsive in the Flink slack - I talked to him quite a bit there. > > If there is a pressing need to start earlier, we can also cut the 3.4 > branch (which is then effectively the 3.3 branch) earlier and backport the > lineage PR (it's just one commit ultimately). I'd leave that decision to > the two release managers for 4.0.0 mentioned before. > > One thing to note for 4.0.0 is that we need to solve the transaction > management issues with the KafkaSink [2]. It's blocking larger users from > adopting the KafkaSink which will be the only option for Flink 2.0. I have > started designing a solution. > > Best, > > Arvid > > > [1] https://github.com/apache/flink-connector-kafka/pull/130 > [2] https://issues.apache.org/jira/browse/FLINK-34554 > > On Tue, Nov 5, 2024 at 4:48 AM Yanquan Lv <decq12y...@gmail.com> wrote: > >> Hi, Arvid. >> >> It has been a month and we are glad to see that we have completed the >> release of Kafka 3.3.0 targeting 1.19 and 1.20. >> >> Considering that Flink 2.0-preview1 has already been released, I would >> like to know about our plans and progress for bumping to 2.0-preview1. >> I tested the changes required for bump to 2.0-preview1 locally and found >> that the adaptation changes made in the production code based on >> FlinkKafkaProducer Depreciated work were relatively clear and the amount of >> change was not significant. However, the headache was that there were many >> adjustments needed in the test code. >> >> I would like to know if there is already work in the community to bump to >> 2.0-preview1. If not, I can help complete this task (but some suggestions >> may be needed for testing the adaptation in the code). >> >> >> >> >> >>> 2024年9月27日 16:23，Arvid Heise <ar...@apache.org> 写道： >>> >>> Dear Flink devs, >>> >>> I'd like to initiate three(!) Kafka connector releases. The main reason >> for >>> having three releases is that we have been slacking a bit in keeping up >>> with the latest changes. >>> >>> Here is the summary: >>> 1. Release kafka-3.3.0 targeting 1.19 and 1.20 (asap) >>> - Incorporates lots of deprecations for Flink 2 including everything that >>> is related to FlinkKafkaProducer (SinkFunction, FlinkKafkaConsumer >>> (SourceFunction), and KafkaShuffle >>> - Lots of bugfixes that are very relevant for 1.19 users (and probably >> work >>> with older releases) >>> >>> 2. Release kafka-3.4.0 targeting 1.20 (~1-2 weeks later) >>> - Incorporates lineage tracing which is only available in 1.20 [1] (FLIP >>> incorrectly says that it's avail in 1.19) >>> - We have discussed some alternatives to this release in [2] but >> basically >>> having a separate release is the cleanest solution. >>> - I'd like to linearize the releases to avoid having to do back or even >>> forward ports >>> >>> 3. Release kafka-4.0.0 targeting 2.0-preview (~1-2 weeks later) >>> - Much requested to get the connector out asap for the preview. (I think >>> the old jar using the removed interfaces should still work) >>> - Remove all deprecated things >>> - General spring cleaning (trying to get rid of arch unit violations, >>> migrate to JUnit5) >>> - Should we relocate the TableAPI stuff to o.a.f.connectors? >>> >>> I'd appreciate any feedback and volunteers for RM ;) If you have pending >>> PRs that should be part of any of those releases, please also write them. >>> >>> Best, >>> >>> Arvid >>> >>> [1] >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-314%3A+Support+Customized+Job+Lineage+Listener >>> [1] >>> >> https://github.com/apache/flink-connector-kafka/pull/111#issuecomment-2306382878 >> >>
