Hi Qingsheng,
Thanks For the feedback.
Other metrics mentioned in FLIP-33, we will also implement it.
Best,
Jiabao
------------------------------------------------------------------
From:Qingsheng Ren <renqs...@gmail.com>
Send Time:2022年9月28日(星期三) 18:36
To:孙家宝 <jiabao....@xtransfer.cn>
Cc:dev <dev@flink.apache.org>
Subject:Re: [DISCUSS] FLIP-262 MongoDB Connector
Thanks for the update Jiabao.
c. It’ll be nice to have all metrics defined in FLIP-33 in the connector. Also
it’s better to have some MongoDB specific metrics, which will be very helpful
for users to monitor their jobs.
Best,
Qingsheng
On Sep 27, 2022, at 11:59, Jiabao Sun <jiabao....@xtransfer.cn
<mailto:jiabao....@xtransfer.cn >> wrote:
Hi Qingsheng,
Thanks for the great suggestion.
a. Currently we have implemented the Source of FLIP-177 in
https://github.com/apache/flink/pull/20848
<https://github.com/apache/flink/pull/20848 >.
Sorry for some mistakes in the Public Interface of FLIP-262, the interface of
Sink has been adjusted from FLIP-143 to FLIP-177.
b. The AsyncWriter mentioned by FLIP-171 looks helpful, I'll try to implement
it.
c. We have provided some common metrics mentioned by FLIP-33.
Source: numRecordsSend
Sink: numRecordsSend/currentSendTime
Best,
Jiabao
------------------------------------------------------------------
From:Qingsheng Ren <re...@apache.org <mailto:re...@apache.org >>
Send Time:2022年9月27日(星期二) 10:43
To:dev <dev@flink.apache.org <mailto:dev@flink.apache.org >>
Cc:孙家宝 <jiabao....@xtransfer.cn <mailto:jiabao....@xtransfer.cn >>
Subject:Re: [DISCUSS] FLIP-262 MongoDB Connector
Hi Jiabao,
Thanks for the FLIP! Very glad to see having a new connector in Flink
eco-system, so overall +1 from my side.
In addition to those FLIPs you mention in the proposal, here’s some other FLIPs
that you may be interested in:
a. FLIP-177 [1] extended Sink API by Sink API v2 with some additional features,
so it’ll be great to implement the sink directly on the latest interface.
b. FLIP-171 [2] introduced async sink that you can to reuse if at-least-once
semantic could fulfill the requirement of MongoDB sink.
c. FLIP-33 [3] defined general source and sink metrics that connectors should
report.
Looking forward to the new connector!
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-177%3A+Extend+Sink+API
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-177%3A+Extend+Sink+API >
[2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink >
[3]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics
>
Best,
Qingsheng
On Sep 23, 2022, at 00:01, 孙家宝 <jiabao....@xtransfer.cn.INVALID
<mailto:jiabao....@xtransfer.cn.INVALID >> wrote:
Hi everyone,
I would like to start a discussion thread on FLIP-262 for a MongoDB Connector
that will support both
DataStream, Table API and SQL implementations with Lookup ability.
This would be a new connector in its own external repository being
https://www.github.com/apache/flink-connector-mongodb
<https://www.github.com/apache/flink-connector-mongodb >
<https://www.github.com/apache/flink-connector-mongodb
<https://www.github.com/apache/flink-connector-mongodb > >.
This repository doesn't exist yet, so we created a FLIP to discuss whether we
need to create it.
https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector
>
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector
> >
Looking forward to any comments or feedback.
Best Regards,
Jiabao
