Hello, I'm super excited that Flink CDC will join the Flink project! It will be so much easier to bring it into Flink-based projects we run.
As I understand you're thinking with Flink CDC 3.0 to make the project broader than CDC, as more "Streaming Integration" framework. This will resemble Kafka Connect, what about renaming the project to something like "Flink Connect"? :-) It would be amazing to have such a tool, "swiss army knife" for data integration, based on Flink. Best Krzysztof On Tue, Dec 12, 2023 at 3:01 PM Lincoln Lee <lincoln.8...@gmail.com> wrote: > +1 > Thanks Leonard and all CDC contributors for the great work! > > Best, > Lincoln Lee > > > Dian Fu <dian0511...@gmail.com> 于2023年12月12日周二 19:11写道： > > > Big +1 for this! Thanks for driving this effort! > > > > Regards, > > Dian > > > > On Mon, Dec 11, 2023 at 4:48 PM Leonard Xu <xbjt...@gmail.com> wrote: > > > > > > Hey all, > > > > > > I want to extend my gratitude for the positive feedback we've received, > > particularly from voices in other communities. It is an honor for the > Flink > > CDC community to have received your recognition. > > > > > > Moving forward, I will collaborate with the Flink CDC community > > maintainers to prepare a FLIP (Flink Improvement Proposal) document that > > will outline specific plans and implementation details. > > > > > > Thank you for your support, and we look forward to contributing to > > Apache Flink community. > > > > > > Best, > > > Leonard > > > > > > > 2023年12月11日 下午4:12，yh z <zhengyunhon...@gmail.com> 写道： > > > > > > > > Excited to hear the news, > > > > +1 > > > > > > > > Best, > > > > Yunhong (Swuferhong) > > > > > > > > liu ron <ron9....@gmail.com> 于2023年12月11日周一 14:05写道： > > > > > > > >> +1 > > > >> > > > >> Best, > > > >> Ron > > > >> > > > >> Yunqing Mo <yunq...@apache.org> 于2023年12月11日周一 12:01写道： > > > >> > > > >>> So cool, Big +1 for this exciting work. > > > >>> > > > >>> On 2023/12/07 03:24:59 Leonard Xu wrote: > > > >>>> Dear Flink devs, > > > >>>> > > > >>>> As you may have heard, we at Alibaba (Ververica) are planning to > > donate > > > >>> CDC Connectors for the Apache Flink project[1] to the Apache Flink > > > >>> community. > > > >>>> > > > >>>> CDC Connectors for Apache Flink comprise a collection of source > > > >>> connectors designed specifically for Apache Flink. These > > connectors[2] > > > >>> enable the ingestion of changes from various databases using Change > > Data > > > >>> Capture (CDC), most of these CDC connectors are powered by > > Debezium[3]. > > > >>> They support both the DataStream API and the Table/SQL API, > > facilitating > > > >>> the reading of database snapshots and continuous reading of > > transaction > > > >>> logs with exactly-once processing, even in the event of failures. > > > >>>> > > > >>>> > > > >>>> Additionally, in the latest version 3.0, we have introduced many > > > >>> long-awaited features. Starting from CDC version 3.0, we've built a > > > >>> Streaming ELT Framework available for streaming data integration. > > This > > > >>> framework allows users to write their data synchronization logic > in a > > > >>> simple YAML file, which will automatically be translated into a > Flink > > > >>> DataStreaming job. It emphasizes optimizing the task submission > > process > > > >> and > > > >>> offers advanced functionalities such as whole database > > synchronization, > > > >>> merging sharded tables, and schema evolution[4]. > > > >>>> > > > >>>> > > > >>>> I believe this initiative is a perfect match for both sides. For > the > > > >>> Flink community, it presents an opportunity to enhance Flink's > > > >> competitive > > > >>> advantage in streaming data integration, promoting the healthy > > growth and > > > >>> prosperity of the Apache Flink ecosystem. For the CDC Connectors > > project, > > > >>> becoming a sub-project of Apache Flink means being part of a > neutral > > > >>> open-source community, which can attract a more diverse pool of > > > >>> contributors. > > > >>>> > > > >>>> Please note that the aforementioned points represent only some of > > our > > > >>> motivations and vision for this donation. Specific future > operations > > need > > > >>> to be further discussed in this thread. For example, the > sub-project > > name > > > >>> after the donation; we hope to name it Flink-CDC aiming to > streaming > > data > > > >>> intergration through Apache Flink, following the naming convention > of > > > >>> Flink-ML; And this project is managed by a total of 8 maintainers, > > > >>> including 3 Flink PMC members and 1 Flink Committer. The remaining > 4 > > > >>> maintainers are also highly active contributors to the Flink > > community, > > > >>> donating this project to the Flink community implies that their > > > >> permissions > > > >>> might be reduced. Therefore, we may need to bring up this topic for > > > >> further > > > >>> discussion within the Flink PMC. Additionally, we need to discuss > > how to > > > >>> migrate existing users and documents. We have a user group of > nearly > > > >> 10,000 > > > >>> people and a multi-version documentation site need to migrate. We > > also > > > >> need > > > >>> to plan for the migration of CI/CD processes and other specifics. > > > >>>> > > > >>>> > > > >>>> While there are many intricate details that require > implementation, > > we > > > >>> are committed to progressing and finalizing this donation process. > > > >>>> > > > >>>> > > > >>>> Despite being Flink’s most active ecological project (as evaluated > > by > > > >>> GitHub metrics), it also boasts a significant user base. However, I > > > >> believe > > > >>> it's essential to commence discussions on future operations only > > after > > > >> the > > > >>> community reaches a consensus on whether they desire this donation. > > > >>>> > > > >>>> > > > >>>> Really looking forward to hear what you think! > > > >>>> > > > >>>> > > > >>>> Best, > > > >>>> Leonard (on behalf of the Flink CDC Connectors project > maintainers) > > > >>>> > > > >>>> [1] https://github.com/ververica/flink-cdc-connectors > > > >>>> [2] > > > >>> > > > >> > > > https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-connectors.html > > > >>>> [3] https://debezium.io > > > >>>> [4] > > > >>> > > > >> > > > https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-pipeline.html > > > >>> > > > >> > > > > > >
