Re: [Discuss][DSIP] Support load binlog from MySQL directly without relying other service

[Xu,Yang(INF)]

Yes. That is my thought too. The best case is to integrate the existing binlog 
load into the routine load.
But we need to find a way to parse binlog in be.

在 2022/3/30 上午11:49，“蔡聪辉”<caiconghui2...@163.com> 写入:

    I think one of the elegant implementations may be something like routine 
load

    在 2022-03-30 10:47:39，"ling miao" <lingm...@apache.org> 写道：
    >> Do you mean 'canal' component?
    >Yes.
    >
    >It sounds like you mean a new way about binlog load?
    >You can show the architecture diagram and general design on the wiki, so
    >that I can better understand your intentions.[1]
    >
    >Ling Miao
    >[
    
>https://cwiki.apache.org/confluence/display/DORIS/DSIP-003%3A+Directly+binlog+consumption
    >]
    >
    >晖 <827677...@qq.com.invalid> 于2022年3月29日周二 20:56写道：
    >
    >> Hi Ling,&nbsp;
    >> Do you mean 'canal' component? I have not cancelled any component,&nbsp;
    >> only extended current binlog load with a new type of 'debezium'.
    >>
    >>
    >> Because the Canal service is not provided in my company's product
    >> environment,
    >> users can't use the current binlog load. I think the new type simplifies
    >> the usage
    >> of binlog load, and shortens the data flow.
    >>
    >>
    >> The similiar function is provided by ClickHouse. I think it's a brilliant
    >> experience to
    >> use a MPP database as a MySQL's slave. It's not the same background with
    >> Flink&nbsp;
    >> Connector.
    >>
    >>
    >>
    >>
    >> ------------------&nbsp;原始邮件&nbsp;------------------
    >> 发件人:
    >>                                                   "dev"
    >>                                                                 <
    >> lingm...@apache.org&gt;;
    >> 发送时间:&nbsp;2022年3月29日(星期二) 晚上8:13
    >> 收件人:&nbsp;"dev"<dev@doris.apache.org&gt;;
    >>
    >> 主题:&nbsp;Re: [Discuss][DSIP] Support load binlog from MySQL directly
    >> without relying other service
    >>
    >>
    >>
    >> Hi hui,
    >>
    >> I want to know what is the reason for canceling the cancel component? Is
    >> there any problem in use? What are the benefits ?
    >>
    >> Parsing binlog is not actually the function of Doris itself. Or is it
    >> better to appear as a connector? For example, like Flink connector.
    >>
    >> Ling Miao
    >>
    >> 陈明雨 <morning...@163.com&gt; 于2022年3月29日周二 19:46写道：
    >>
    >> &gt; Hi Hui,
    >> &gt; Thanks for it. Could you register a cwiki acount and tell me your
    >> account
    >> &gt; id? So that I can create DSIP and grant write priv for you.
    >> &gt; https://cwiki.apache.org/confluence/signup.action
    >> &gt;
    >> &gt;
    >> &gt;
    >> &gt;
    >> &gt; --
    >> &gt;
    >> &gt; 此致！Best Regards
    >> &gt; 陈明雨 Mingyu Chen
    >> &gt;
    >> &gt; Email:
    >> &gt; chenmin...@apache.org
    >> &gt;
    >> &gt;
    >> &gt;
    >> &gt;
    >> &gt;
    >> &gt; At 2022-03-29 18:17:02, "晖" <827677...@qq.com.INVALID&gt; wrote:
    >> &gt; &gt;As issue #8025(
    >> &gt;
    >> 
https://github.com/apache/incubator-doris/issues/8025)&amp;nbsp;mentioned,
    >> &gt; the current binlog load is relying on Canal service. We can 
integrate
    >> the
    >> &gt; binlog aquire and parse function into Doris, so that user can be 
more
    >> &gt; convinience to use it.Please determine the final DSIP number for it.
    >> &gt;