Randall, I plan to update the public details section and the performance impact as you recommended.
Regarding state-specific counts, I do agree this is a useful addition. Before I make the change, I'd like to agree that these state-specific counts should be in addition to the already-proposed total tasks count (even though might be redundant, it is robust against new/missed connector states, and is a useful metric in its own right), yes? Cyrus On Tue, Jun 4, 2019 at 12:24 PM Randall Hauch <rha...@gmail.com> wrote: > Thanks, Cyrus -- this will be quite useful. I do have a few > comments/requests. > > Can you please be more specific about the public details about the metric? > What is the MBean name on which the metric will appear? For example, the AK > documentation (https://kafka.apache.org/documentation/#connect_monitoring) > defines all of the metrics an where they will appear, as does > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-196%3A+Add+metrics+to+Kafka+Connect+framework > . > > Secondly, while a metric showing the total number of tasks is very useful, > might it be worth considering also adding metrics for the number of running > tasks, the number of paused tasks, and the number of failed tasks for a > connector. It might require using the herder's `connectorStatus(String > connectorName)` method instead, but that appears to be just as effective at > using the local snapshot of the status store cache. > > Thirdly, it might be useful for the KIP to address the potential > performance impact of computing these methods. Again, IIUC, the herder > methods that the proposal mentions use the status and config stores caches > only, so the impact should be negligible. > > Best regards, > > Randall > > On Sun, Jun 2, 2019 at 10:05 PM Ryanne Dolan <ryannedo...@gmail.com> > wrote: > > > Cyrus, I agree this would be useful. > > > > Ryanne > > > > On Fri, May 31, 2019, 7:10 PM Oleksandr Diachenko <odiache...@apache.org > > > > wrote: > > > > > > > > > > > On 2019/05/30 06:06:12, Cyrus Vafadari <cy...@confluent.io> wrote: > > > > Hello Dev, > > > > > > > > I'd like to start the discussion of KIP-475: New Metric to Measure > > Number > > > > of Tasks on a Connector. > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-475%3A+New+Metric+to+Measure+Number+of+Tasks+on+a+Connector > > > > > > > > The proposal is pretty straightforward -- to add a new metric to > > Connect > > > to > > > > measure the number of tasks on a Connector. Currently, we support > this > > on > > > > Worker level, so this KIP just adds another metric to support this > > > > per-connector. > > > > > > > > There is also a PR: > > > > https://github.com/apache/kafka/pull/6843 > > > > > > > > Thanks, > > > > > > > > Cyrus > > > > > > > > > > Hi Cyrus, > > > > > > That sounds like a useful addition. > > > > > > Regards, Alex. > > > > > >
