Thanks, Gwen. That's a great idea, so I've changed the KIP to add those metrics. I've also made a few other changes:
1. The context of all metrics is limited to the activity within the worker. This wasn't clear before, so I changed the motivation and metric descriptions to explicitly state this. 2. Added the worker ID to all MBean attributes. In addition to hopefully making this same scope obvious from within JMX or other metric reporting system. This is also similar to how the Kafka producer and consumer metrics include the client ID in their MBean attributes. Hopefully this does not negatively impact or complicate how external reporting systems' aggregate metrics from multiple workers. 3. Stated explicitly that aggregating metrics across workers was out of scope of this KIP. 4. Added metrics to report the connector class and version for both sink and source connectors. Check this KIP's history for details of these changes. Please let me know if you have any other suggestions. I hope to start the voting soon! Best regards, Randall On Thu, Sep 7, 2017 at 9:35 PM, Gwen Shapira <g...@confluent.io> wrote: > Thanks for the KIP, Randall. Those are badly needed! > > Can we have two metrics with record rate per task? One before SMT and one > after? > We can have cases where we read 5000 rows from JDBC but write 5 to Kafka, > or read 5000 records from Kafka and write 5 due to filtering. I think its > important to know both numbers. > > > Gwen > > On Thu, Sep 7, 2017 at 7:50 PM, Randall Hauch <rha...@gmail.com> wrote: > > > Hi everyone. > > > > I've created a new KIP to add metrics to the Kafka Connect framework: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 196%3A+Add+metrics+to+Kafka+Connect+framework > > > > The KIP approval deadline is looming, so if you're interested in Kafka > > Connect metrics please review and provide feedback as soon as possible. > I'm > > interested not only in whether the metrics are sufficient and > appropriate, > > but also in whether the MBean naming conventions are okay. > > > > Best regards, > > > > Randall > > > > > > -- > *Gwen Shapira* > Product Manager | Confluent > 650.450.2760 | @gwenshap > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog > <http://www.confluent.io/blog> >
