Thanks for the feedback, Shaoxuan. I'll create a FLIP then. Good point about documentation. I'll put together a doc on the website once we agreed upon the detail metric specifications.
Jiangjie (Becket) Qin On Mon, Feb 11, 2019 at 4:10 PM Shaoxuan Wang <wshaox...@gmail.com> wrote: > Becket, > > Thanks for the proposal. I am much agree that we should only design the > standard/common Flink-related connector metrics, while keep the original > names for the ones provided by the connector itself. If everyone agrees to > make these changes, we should do it asap, the earlier to make the change > the less impact on the user's system. Maybe we can start to unify the > metrics on the existing connectors, and then slowly introduce the new > common/standard metrics if necessary. > > Since metric is also a kind of user interface, we should either open a > Flip, or create a webpage to document the specifications, it's up to you. > The point is that we want to create a good document, such that the future > developers can easily find and follow these specifications. > > Regards, > Shaoxuan > > On Fri, Feb 1, 2019 at 5:15 PM Becket Qin <becket....@gmail.com> wrote: > > > Thanks for the feedback, Konstantin, > > > > Completely agree that exposing the original metrics from the clients of > > other systems is a good practice and should be encouraged. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > On Fri, Feb 1, 2019 at 4:01 PM Konstantin Knauf < > > konstan...@da-platform.com> > > wrote: > > > > > Hi Becket, > > > > > > I like the idea of providing a standard set of metrics, which sources > can > > > choose to implement/expose. In addition, I think, sources, like the > Kafka > > > or Kinesis Source, should continue to forward the original consumer > > metrics > > > under their original names, so that users familiar with Kafka/Kinesis > can > > > relate to them easily. > > > > > > Cheers, > > > > > > Konstantin > > > > > > > > > > > > On Fri, Feb 1, 2019 at 3:54 AM Becket Qin <becket....@gmail.com> > wrote: > > > > > > > Thanks for the connector metric url, Chesnay :) > > > > > > > > @Robert, as you can see, the metrics from different connectors are > > quite > > > > different. And there are different names for similar metrics, which > is > > a > > > > little frustrating when users want to do monitoring / alerting. > > > > > > > > Thanks, > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > On Thu, Jan 31, 2019 at 6:41 PM Chesnay Schepler <ches...@apache.org > > > > > > wrote: > > > > > > > > > @Robert: > > > > > > > > > > > > > > > > > > > > https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#connectors > > > > > > > > > > On 31.01.2019 11:03, Robert Metzger wrote: > > > > > > Hey Becket, > > > > > > thanks a lot for your proposal! > > > > > > > > > > > > Do you have an overview over the current situation of the metrics > > in > > > > the > > > > > > connectors? > > > > > > Which connectors expose metrics at all? > > > > > > Are they different? > > > > > > > > > > > > On Thu, Jan 31, 2019 at 8:44 AM Becket Qin <becket....@gmail.com > > > > > > wrote: > > > > > > > > > > > >> Hi folks, > > > > > >> > > > > > >> I was trying to add some metrics to Kafka connectors and > realized > > > that > > > > > >> right now Flink does not have a common metric definition for the > > > > > >> connectors. This complicates the monitoring and operation > because > > > the > > > > > >> monitoring / alerts need to be set case by case. > > > > > >> > > > > > >> To address this issue, I would like to see if is possible to > have > > a > > > > set > > > > > of > > > > > >> standardized common metrics for all sources and sinks. The > > following > > > > doc > > > > > >> describes the proposal. Feedback is very welcome. > > > > > >> > > > > > >> > > > > > >> > > > > > > > > > > > > > > > https://docs.google.com/document/d/1q86bgj_3T6WFbSUoxLDJJXmUcBOUcvWfh2RZvHG-nPU/edit# > > > > > >> > > > > > >> Thanks, > > > > > >> > > > > > >> Jiangjie (Becket) Qin > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > -- > > > -- > > > > > > Konstantin Knauf | Solution Architect > > > > > > > > > -- > > > > > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > > > Conference > > > > > > Stream Processing | Event Driven | Real Time > > > > > > -- > > > > > > Data Artisans GmbH | Stresemannstr. 121A,10963 Berlin, Germany > > > < > > > > > > https://maps.google.com/?q=Stresemannstr.+121A,10963+Berlin,+Germany&entry=gmail&source=g > > > > > > > data Artisans, Inc. | 1161 Mission Street, San Francisco, CA-94103, USA > > > < > > > > > > https://maps.google.com/?q=1161+Mission+Street,+San+Francisco,+CA-94103,+USA&entry=gmail&source=g > > > > > > > > > > -- > > > Data Artisans GmbH > > > Registered at Amtsgericht Charlottenburg: HRB 158244 B > > > Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen > > > > > >
