Thanks for the update Jiabao. c. It’ll be nice to have all metrics defined in FLIP-33 in the connector. Also it’s better to have some MongoDB specific metrics, which will be very helpful for users to monitor their jobs.
Best, Qingsheng > On Sep 27, 2022, at 11:59, Jiabao Sun <jiabao....@xtransfer.cn > <mailto:jiabao....@xtransfer.cn>> wrote: > > Hi Qingsheng, > > Thanks for the great suggestion. > > a. Currently we have implemented the Source of FLIP-177 in > https://github.com/apache/flink/pull/20848 > <https://github.com/apache/flink/pull/20848>. > Sorry for some mistakes in the Public Interface of FLIP-262, the interface > of Sink has been adjusted from FLIP-143 to FLIP-177. > > b. The AsyncWriter mentioned by FLIP-171 looks helpful, I'll try to implement > it. > > c. We have provided some common metrics mentioned by FLIP-33. > Source: numRecordsSend > Sink: numRecordsSend/currentSendTime > > Best, > Jiabao > > ------------------------------------------------------------------ > From:Qingsheng Ren <re...@apache.org <mailto:re...@apache.org>> > Send Time:2022年9月27日(星期二) 10:43 > To:dev <dev@flink.apache.org <mailto:dev@flink.apache.org>> > Cc:孙家宝 <jiabao....@xtransfer.cn <mailto:jiabao....@xtransfer.cn>> > Subject:Re: [DISCUSS] FLIP-262 MongoDB Connector > > Hi Jiabao, > > Thanks for the FLIP! Very glad to see having a new connector in Flink > eco-system, so overall +1 from my side. > > In addition to those FLIPs you mention in the proposal, here’s some other > FLIPs that you may be interested in: > > a. FLIP-177 [1] extended Sink API by Sink API v2 with some additional > features, so it’ll be great to implement the sink directly on the latest > interface. > b. FLIP-171 [2] introduced async sink that you can to reuse if at-least-once > semantic could fulfill the requirement of MongoDB sink. > c. FLIP-33 [3] defined general source and sink metrics that connectors should > report. > > Looking forward to the new connector! > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-177%3A+Extend+Sink+API > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-177%3A+Extend+Sink+API> > [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink> > [3] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics> > > Best, > Qingsheng > > >> On Sep 23, 2022, at 00:01, 孙家宝 <jiabao....@xtransfer.cn.INVALID >> <mailto:jiabao....@xtransfer.cn.INVALID>> wrote: >> >> Hi everyone, >> I would like to start a discussion thread on FLIP-262 for a MongoDB >> Connector that will support both >> DataStream, Table API and SQL implementations with Lookup ability. >> This would be a new connector in its own external repository being >> https://www.github.com/apache/flink-connector-mongodb >> <https://www.github.com/apache/flink-connector-mongodb> >> <https://www.github.com/apache/flink-connector-mongodb >> <https://www.github.com/apache/flink-connector-mongodb> >. >> This repository doesn't exist yet, so we created a FLIP to discuss whether >> we need to create it. >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector >> >> <https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector> >> >> <https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector >> >> <https://cwiki.apache.org/confluence/display/FLINK/FLIP-262%3A+Introduce+MongoDB+connector> >> > >> Looking forward to any comments or feedback. >> Best Regards, >> Jiabao >
