Re: [Discuss][DSIP] Improve Doris observability

[zhg yang]

Hi stalary
 luozenglin <https://github.com/luozenglin> has done a lot of work on
OpenTelemetry and trace before, and has made some progress, I think you can
work with him to develop this thing together
Thanks
Yang Zhengguo


stalary <stal...@163.com> 于2022年5月15日周日 22:55写道：

> I will split it for five steps, step1 will be visualized first.
> Step1. Sql Plan
> Step2. Execute Profile and data integration
> Step3. Doris topology
> Step4. Doris fe and be metrics
> Step5. Doris log
>
>
> by stalary
>
>
> On 05/15/2022 22:27，Zhou Minghong<minghong.z...@163.com> wrote：
> Hi Stalary,
> What kind of data will be visualized first?
> Could you integrate profile data and sql plan together？
> Any way, this is a good idea!
>
>
> Minghong
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> At 2022-05-15 19:26:05, "41108453" <41108...@qq.com.INVALID> wrote:
> nice work！
>
>
>
>
>
>
>
> ------------------ Original ------------------
> From: stalary <stal...@163.com&gt;
> Date: Sun,May 15,2022 7:13 PM
> To: dev@doris.apache.org <dev@doris.apache.org&gt;
> Subject: Re: [Discuss][DSIP] Improve Doris observability
>
>
>
>
> Dear All,
>
> I’d like to improve Doris observability. This is achieved by introducing
> Skywalking and OpenTelemetry. Parameters allow users to decide which
> monitoring tool to use. First I want to support trace on the FE, implement
> tracing of SQL statements.
>
> The associated issue：https://github.com/apache/incubator-doris/issues/9577
>
> Here are the results of my preliminary research
> 1. Currently OpenTelemetry is very convincing in terms of protocol
> specifications, including traces, metrics, logs and so on, But it does not
> provide native data visualization, we need to export data to Zipkin or
> Prometheus to visualize.
>
>
> 2. Skywalking is currently popular, active, and has excellent feature
> support, including data visualization, traces, metrics, logs and so on, in
> particular, there is excellent support for data visualization
>
>
> 3. Both have Java and c++ sdk that Doris can use
>
> —
> Best regards
> Stalary
>