Hey Joao, Thanks for kicking off this thread, I have some comments as follows:
> Currently, Flink CDC uses Debezium 1.9, which is the last version > compatible with Java 8. While an upgrade to Flink 2.0 will likely enable a > move to a newer version, we would still be constrained to Debezium 2.7 (the > last version supporting Java 11). This is already part of Flink CDC's roadmap — we've discussed upgrading Flink to version 2.0 [1]. However, like other independent connector repositories, we prefer to wait until Flink 2.0 becomes more stable before bumping both the Flink version and the Debezium version to 2.7. > With the main Debezium project now at > version 3.2 (requiring Java 17), the Flink community is unable to leverage > many valuable evolutions. These features may not be critical to the core In this case, we will still encounter the Java 17 issue regardless of whether we choose to decouple the Debezium connector from Flink or upgrade the Debezium version within Flink CDC, since both approaches depend on Flink, which currently does not support Java 17, right? So why don't we directly accelerate the upgrade to Flink 2.0 and Debezium 2.7 in the Flink CDC project? To be honest, I'm not particularly in favor of splitting the connectors into external projects and releasing them independently. (1) Frankly, Flink's external connectors currently lack sufficient community support for maintenance. For instance, HBase is a sufficiently influential project within the big data ecosystem. However, the latest supported Flink version is 1.19.x, which was released last year [2]. I've seen contributors in the community pushing for adaptation to Flink 2.0. Without enough dedicated committers and PMCs willing to invest time and effort into developing and maintaining these connectors, their future development cannot be guaranteed. (2) The Flink CDC project has been steadily releasing versions and fixing various issues [3]. This is because contributors to the Flink CDC project, including myself, come from multiple companies and use Flink CDC extensively in our respective organizations. We have the motivation and resources to invest in this project, enabling its healthy growth. (3) Currently, the primary use case for flink-connector-debezium is still within the various CDC connectors in the Flink CDC project. I haven't seen many other usage scenarios. Even if there are, as mentioned in my previous response, the motivation to drive solutions would likely be stronger within the Flink CDC project itself. Overall, I still prefer concentrating the community's limited resources on jointly maintaining a single repository. I don't want to see situations where Flink is already planning for version 2.2, while certain community-maintained connectors remain stuck on version 1.16. Best, Leonard [1]https://lists.apache.org/thread/n7zq6yt31s9xy77wrqtv2wxdn6gv5ytm [2] https://github.com/apache/flink-connector-hbase [3] https://github.com/apache/flink-cdc/releases
