Hi, Hu,
Currently, records-lag-max is an attribute for the mbean
kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{client-id}".
So, it probably makes sense for records-lead-min to be an attribute under
the same mbean.
The partition level records-lead can probably be an attribute for the mbean
kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{client-id}",topic=topic1,partition=0,
where topic and partition are the tags. This matches the topic level mbeans
that we have in the consumer. I am not sure what the per partition level
records-lead-min and records-lead-avg are. Are they the min/avg of the lead
since the consumer is started? I am not sure we need those since an
external monitoring system can always derive them from records-lead.
Thanks,
Jun
On Mon, Nov 13, 2017 at 8:10 PM, Hu Xi <huxi...@hotmail.com> wrote:
> Jun,
>
> Thanks for the feedback. Some things need to make sure. Currently, these
> new-added metrics follow the exact naming convention with those 'lag'
> counterparts, as shown below:
>
>
> Consumer-level metric:
>
> records-lag-max ==> records-lead-min
>
>
> Partition-level metrics:
>
> <topic>-<partitionId>.records-lag ==> <topic>-<partitionId>.
> records-lead
>
> <topic>-<partitionId>.records-lag-max ==> <topic>-<partitionId>.
> records-lead-min
>
> <topic>-<partitionId>.records-lag-avg ==> <topic>-<partitionId>.
> records-lead-avg
>
>
> Correct me if I am wrong, but what you mentioned `*records-lead-avg and
> records-lead-min don't need the partition prefix since they are aggregates
> across all partitions*` seemed to break the naming rule above. Do we
> still have to keep the same rule with the "lag" metrics?
>
>
> huxihx
>
>
>
>
>
> ------------------------------
> *发件人:* Jun Rao <j...@confluent.io>
> *发送时间:* 2017年11月14日 1:48
> *收件人:* dev@kafka.apache.org
> *主题:* Re: [DISCUSS]KIP-223 - Add per-topic min lead and per-partition
> lead metrics to KafkaConsumer
>
> Hi, Hu,
>
> Thanks for the KIP. Looks good overall. Could you document the mbean name
> for the new metrics? We probably want the name to be consistent with
> records-max-lag as described in
> http://kafka.apache.org/documentation/#monitoring. Also, it seems that
> <http://kafka.apache.org/documentation/#monitoring>
> Apache Kafka <http://kafka.apache.org/documentation/#monitoring>
> kafka.apache.org
> 1.2 Use Cases. Here is a description of a few of the popular use cases for
> Apache Kafka®. For an overview of a number of these areas in action, see
> this blog post.
>
>
> records-lead-avg and records-lead-min don't need the partition prefix since
> they are aggregates across all partitions. For records-lead, it seems that
> it's better to add the topic partition as a tag, instead of as a prefix in
> the metric name.
>
> Jun
>
>
>
>
> On Thu, Nov 9, 2017 at 1:03 AM, Hu Xi <huxi...@hotmail.com> wrote:
>
> > Hi all,
> >
> >
> > As per Jun Rao's suggestion, I opened up the KIP-223(
> https://cwiki.apache.
> > org/confluence/display/KAFKA/KIP-223+-+Add+per-topic+min+
> > lead+and+per-partition+lead+metrics+to+KafkaConsumer) concerning adding
> > new kinds of lag metrics for KafkaConsumer. Be free to leave your
> comments
> > here. Thanks in advance.
> >
> >
>
