Hi Leonard, Thank you for the excellent work you and the team working on the CDC connectors project have been doing so far. I am +1 of having them under Flink's umbrella.
On Thu, Dec 7, 2023 at 10:26 AM Etienne Chauchot <echauc...@apache.org> wrote: > Big +1, thanks this will be a very useful addition to Flink. > > Best > > Etienne > > Le 07/12/2023 à 09:26, Hang Ruan a écrit : > > +1 for contributing CDC Connectors to Apache Flink. > > > > Best, > > Hang > > > > Yuxin Tan<tanyuxinw...@gmail.com> 于2023年12月7日周四 16:05写道： > > > >> Cool, +1 for contributing CDC Connectors to Apache Flink. > >> > >> Best, > >> Yuxin > >> > >> > >> Jing Ge<j...@ververica.com.invalid> 于2023年12月7日周四 15:43写道： > >> > >>> Awesome! +1 > >>> > >>> Best regards, > >>> Jing > >>> > >>> On Thu, Dec 7, 2023 at 8:34 AM Sergey Nuyanzin<snuyan...@gmail.com> > >>> wrote: > >>> > >>>> thanks for working on this and driving it > >>>> > >>>> +1 > >>>> > >>>> On Thu, Dec 7, 2023 at 7:26 AM Feng Jin<jinfeng1...@gmail.com> > wrote: > >>>> > >>>>> This is incredibly exciting news, a big +1 for this. > >>>>> > >>>>> Thank you for the fantastic work on Flink CDC. We have created > >>> thousands > >>>> of > >>>>> real-time integration jobs using Flink CDC connectors. > >>>>> > >>>>> > >>>>> Best, > >>>>> Feng > >>>>> > >>>>> On Thu, Dec 7, 2023 at 1:45 PM gongzhongqiang < > >>> gongzhongqi...@apache.org > >>>>> wrote: > >>>>> > >>>>>> It's very exciting to hear the news. > >>>>>> +1 for adding CDC Connectors to Apache Flink ! > >>>>>> > >>>>>> > >>>>>> Best, > >>>>>> Zhongqiang > >>>>>> > >>>>>> Leonard Xu<xbjt...@gmail.com> 于2023年12月7日周四 11:25写道： > >>>>>> > >>>>>>> Dear Flink devs, > >>>>>>> > >>>>>>> > >>>>>>> As you may have heard, we at Alibaba (Ververica) are planning to > >>>> donate > >>>>>> CDC Connectors for the Apache Flink project > >>>>>>> *[1]* to the Apache Flink community. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> CDC Connectors for Apache Flink comprise a collection of source > >>>>>> connectors designed specifically for Apache Flink. These connectors > >>>>>>> *[2]* > >>>>>>> enable the ingestion of changes from various databases using > >>> Change > >>>>>> Data Capture (CDC), most of these CDC connectors are powered by > >>>> Debezium > >>>>>>> *[3]* > >>>>>>> . They support both the DataStream API and the Table/SQL API, > >>>>>> facilitating the reading of database snapshots and continuous > >> reading > >>>> of > >>>>>> transaction logs with exactly-once processing, even in the event of > >>>>>> failures. > >>>>>>> > >>>>>>> > >>>>>>> Additionally, in the latest version 3.0, we have introduced many > >>>>>> long-awaited features. Starting from CDC version 3.0, we've built a > >>>>>> Streaming ELT Framework available for streaming data integration. > >>> This > >>>>>> framework allows users to write their data synchronization logic > >> in a > >>>>>> simple YAML file, which will automatically be translated into a > >> Flink > >>>>>> DataStreaming job. It emphasizes optimizing the task submission > >>> process > >>>>> and > >>>>>> offers advanced functionalities such as whole database > >>> synchronization, > >>>>>> merging sharded tables, and schema evolution > >>>>>>> *[4]*. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> I believe this initiative is a perfect match for both sides. For > >>> the > >>>>>> Flink community, it presents an opportunity to enhance Flink's > >>>>> competitive > >>>>>> advantage in streaming data integration, promoting the healthy > >> growth > >>>> and > >>>>>> prosperity of the Apache Flink ecosystem. For the CDC Connectors > >>>> project, > >>>>>> becoming a sub-project of Apache Flink means being part of a > >> neutral > >>>>>> open-source community, which can attract a more diverse pool of > >>>>>> contributors. > >>>>>>> > >>>>>>> Please note that the aforementioned points represent only some of > >>> our > >>>>>> motivations and vision for this donation. Specific future > >> operations > >>>> need > >>>>>> to be further discussed in this thread. For example, the > >> sub-project > >>>> name > >>>>>> after the donation; we hope to name it Flink-CDC > >>>>>>> aiming to streaming data intergration through Apache Flink, > >>>>>>> following the naming convention of Flink-ML; And this project is > >>>>> managed > >>>>>> by a total of 8 maintainers, including 3 Flink PMC members and 1 > >>> Flink > >>>>>> Committer. The remaining 4 maintainers are also highly active > >>>>> contributors > >>>>>> to the Flink community, donating this project to the Flink > >> community > >>>>>> implies that their permissions might be reduced. Therefore, we may > >>> need > >>>>> to > >>>>>> bring up this topic for further discussion within the Flink PMC. > >>>>>> Additionally, we need to discuss how to migrate existing users and > >>>>>> documents. We have a user group of nearly 10,000 people and a > >>>>> multi-version > >>>>>> documentation site need to migrate. We also need to plan for the > >>>>> migration > >>>>>> of CI/CD processes and other specifics. > >>>>>>> > >>>>>>> > >>>>>>> While there are many intricate details that require > >> implementation, > >>>> we > >>>>>> are committed to progressing and finalizing this donation process. > >>>>>>> > >>>>>>> > >>>>>>> Despite being Flink’s most active ecological project (as > >> evaluated > >>> by > >>>>>> GitHub metrics), it also boasts a significant user base. However, I > >>>>> believe > >>>>>> it's essential to commence discussions on future operations only > >>> after > >>>>> the > >>>>>> community reaches a consensus on whether they desire this donation. > >>>>>>> > >>>>>>> Really looking forward to hear what you think! > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Best, > >>>>>>> Leonard (on behalf of the Flink CDC Connectors project > >> maintainers) > >>>>>>> [1]https://github.com/ververica/flink-cdc-connectors > >>>>>>> [2] > >>>>>>> > >> > https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-connectors.html > >>>>>>> [3]https://debezium.io > >>>>>>> [4] > >>>>>>> > >> > https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-pipeline.html > >>>> > >>>> -- > >>>> Best regards, > >>>> Sergey > >>>>
