One minor comment, is there any other encoding or format in debezium? I'm asking because the format name is debezium-json, i'm wondering whether debezium is enough. This also applies to canal.
Best, Kurt On Tue, Apr 7, 2020 at 11:49 AM Jark Wu <imj...@gmail.com> wrote: > Hi everyone, > > Since this FLIP was proposed, the community has discussed a lot about the > first approach: introducing new TableSource and TableSink interfaces to > support changelog. > And yes, that is FLIP-95 which has been accepted last week. So most of the > work has been merged into FLIP-95. > > In order to support the goal of FLIP-105, there is still a little things to > discuss: how to connect external CDC formats. > We propose to introduce 2 new formats: Debezium format and Canal format. > They are the most popular CDC tools according to the survey in user [1] and > user-zh [2] mailing list. > > I have updated the FLIP: > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-105%3A+Support+to+Interpret+and+Emit+Changelog+in+Flink+SQL > > Welcome feedbacks! > > Best, > Jark > > [1]: > > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/SURVEY-What-Change-Data-Capture-tools-are-you-using-td33569.html > [2]: http://apache-flink.147419.n8.nabble.com/SURVEY-CDC-td1910.html > > > On Fri, 14 Feb 2020 at 22:08, Jark Wu <imj...@gmail.com> wrote: > > > Hi everyone, > > > > I would like to start discussion about how to support interpreting > > external changelog into Flink SQL, and how to emit changelog from Flink > SQL. > > > > This topic has already been mentioned several times in the past. CDC > > (Change Data Capture) data has been a very important streaming data in > the > > world. Connect to CDC is a significant feature for Flink, it fills the > > missing piece for Flink's streaming processing. > > > > In FLIP-105, we propose 2 approaches to achieve. > > One is introducing new TableSource interface (higher priority), > > the other is introducing new SQL syntax to interpret and emit changelog. > > > > FLIP-105: > > > https://docs.google.com/document/d/1onyIUUdWAHfr_Yd5nZOE7SOExBc6TiW5C4LiL5FrjtQ/edit# > > > > Thanks for any feedback! > > > > Best, > > Jark > > >
