Hi Qingsheng,
Thanks for the great suggestion.
a. Currently we have implemented the Source of FLIP-177 in
https://github.com/apache/flink/pull/20848.
Sorry for some mistakes in the Public Interface of FLIP-262, the interface of
Sink has been adjusted from FLIP-143 to FLIP-177.
b. The AsyncWriter mentioned by FLIP-171 looks helpful, I'll try to implement
it.
c. We have provided some common metrics mentioned by FLIP-33.
Source: numRecordsSend
Sink: numRecordsSend/currentSendTime
Best,
Jiabao
------------------------------------------------------------------
From:Qingsheng Ren <re...@apache.org>
Send Time:2022年9月27日(星期二) 10:43
To:dev <dev@flink.apache.org>
Cc:孙家宝 <jiabao....@xtransfer.cn>
Subject:Re: [DISCUSS] FLIP-262 MongoDB Connector
Hi Jiabao,
Thanks for the FLIP! Very glad to see having a new connector in Flink
eco-system, so overall +1 from my side.
In addition to those FLIPs you mention in the proposal, here’s some other FLIPs
that you may be interested in:
a. FLIP-177 [1] extended Sink API by Sink API v2 with some additional features,
so it’ll be great to implement the sink directly on the latest interface.
b. FLIP-171 [2] introduced async sink that you can to reuse if at-least-once
semantic could fulfill the requirement of MongoDB sink.
c. FLIP-33 [3] defined general source and sink metrics that connectors should
report.
Looking forward to the new connector!
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-177%3A+Extend+Sink+API
[2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink
[3]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics
Best,
Qingsheng
> On Sep 23, 2022, at 00:01, 孙家宝 <jiabao....@xtransfer.cn.INVALID> wrote:
>
> Hi everyone,
> I would like to start a discussion thread on FLIP-262 for a MongoDB Connector
> that will support both
> DataStream, Table API and SQL implementations with Lookup ability.
> This would be a new connector in its own external repository being
> https://www.github.com/apache/flink-connector-mongodb
> <https://www.github.com/apache/flink-connector-mongodb >.
> This repository doesn't exist yet, so we created a FLIP to discuss whether we
> need to create it.
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector
>
> <https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector
> >
> Looking forward to any comments or feedback.
> Best Regards,
> Jiabao
