Hi Dawid, Thanks for the feedback. That makes sense to me. There are two cases to be addressed.
1. The metrics are supposed to be a guidance. It is likely that a connector only supports some but not all of the metrics. In that case, each connector implementation should have the freedom to decide which metrics are reported. For the metrics that are supported, the guidance should be followed. 2. Sometimes users may want to disable certain metrics for some reason (e.g. performance / reprocessing of data). A generic mechanism should be provided to allow user choose which metrics are reported. This mechanism should also be honored by the connector implementations. Does this sound reasonable to you? Thanks, Jiangjie (Becket) Qin On Thu, Feb 21, 2019 at 4:22 PM Dawid Wysakowicz <dwysakow...@apache.org> wrote: > Hi, > > Generally I like the idea of having a unified, standard set of metrics for > all connectors. I have some slight concerns about fetchLatency and > latency though. They are computed based on EventTime which is not a purely > technical feature. It depends often on some business logic, might be absent > or defined after source. Those metrics could also behave in a weird way in > case of replaying backlog. Therefore I am not sure if we should include > those metrics by default. Maybe we could at least introduce a feature > switch for them? What do you think? > > Best, > > Dawid > On 21/02/2019 03:13, Becket Qin wrote: > > Bump. If there is no objections to the proposed metrics. I'll start a > voting thread later toady. > > Thanks, > > Jiangjie (Becket) Qin > > On Mon, Feb 11, 2019 at 8:17 PM Becket Qin <becket....@gmail.com> > <becket....@gmail.com> wrote: > > > Hi folks, > > I would like to start the FLIP discussion thread about standardize the > connector metrics. > > In short, we would like to provide a convention of Flink connector > metrics. It will help simplify the monitoring and alerting on Flink jobs. > The FLIP link is following: > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics > > Thanks, > > Jiangjie (Becket) Qin > > >
