Hi hui, I created DSIP-003 for you: https://cwiki.apache.org/confluence/display/DORIS/DSIP-003%3A+Directly+binlog+consumption
And I have some questions: 1. Using debezium can indeed simplify the analysis difficulty of binglog, but the disadvantage is that it occupies the computing resources of FE. This is also the disadvantage of the current Canal solution. So we need some test data to illustrate the resource overhead of this scheme and what synchronization rate can be achieved. 2. What are the advantages and disadvantages of this scheme compared to Canal solution? Is it possible to completely replace Canal's solution? -- 此致！Best Regards 陈明雨 Mingyu Chen Email: chenmin...@apache.org 在 2022-03-29 21:05:08，"晖" <827677...@qq.com.INVALID> 写道： >Hi Yang, > > >I have not found any library we can use. Maybe it's a lot of work to put it on >BE. > > >I think it's not necessary to put it in BE, Because we can distribute the sync >job into >other FE instead of Master, and user can use Flink Connector if there is still >too much data. > > > > >------------------ 原始邮件 ------------------ >发件人: > "dev" > <xuyan...@baidu.com>; >发送时间: 2022年3月29日(星期二) 晚上8:38 >收件人: "dev@doris.apache.org"<dev@doris.apache.org>; > >主题: Re: [Discuss][DSIP] Support load binlog from MySQL directly without >relying other service > > > >Hi hui, >Thanks for the pr. >Is there any way to integrate the binlog aquire and parse function into Be, >instead of Fe? > >在 2022/3/29 下午8:13，“ling miao”<lingm...@apache.org> 写入: > > Hi hui, > > I want to know what is the reason for canceling the cancel >component? Is > there any problem in use? What are the benefits ? > > Parsing binlog is not actually the function of Doris >itself. Or is it > better to appear as a connector? For example, like Flink >connector. > > Ling Miao > > 陈明雨 <morning...@163.com> 于2022年3月29日周二 19:46写道： > > > Hi Hui, > > Thanks for it. Could you register a cwiki acount and >tell me your account > > id? So that I can create DSIP and grant write priv for >you. > > https://cwiki.apache.org/confluence/signup.action > > > > > > > > > > -- > > > > 此致！Best Regards > > 陈明雨 Mingyu Chen > > > > Email: > > chenmin...@apache.org > > > > > > > > > > > > At 2022-03-29 18:17:02, "晖" ><827677...@qq.com.INVALID> wrote: > > >As issue #8025( > > >https://github.com/apache/incubator-doris/issues/8025)&nbsp;mentioned, > > the current binlog load is relying on Canal service. >We can integrate the > > binlog aquire and parse function into Doris, so that >user can be more > > convinience to use it.Please determine the final DSIP >number for it. > > > > >--------------------------------------------------------------------- >To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org >For additional commands, e-mail: dev-h...@doris.apache.org
