These are in regex form for DataDog's JMX collector, but it should get you started:
bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=TotalProduceRequestsPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=TotalFetchRequestsPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=BytesOutPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=FailedFetchRequestsPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=FailedProduceRequestsPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=ProduceMessageConversionsPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=FetchMessageConversionsPerSec,topic=.*' On Wed, Aug 22, 2018 at 9:24 AM Luigi Tagliamonte < luigi.tagliamont...@gmail.com> wrote: > I've got a sandbox cluster (without any control on topic creation) full of > unused topics and I'd like to cleanup all the those topics. > I was able to identify all the topics with 0 messages with the following > lines: > > *#!/bin/bash* > *set -e* > *BROKERS="10.16.208.220:9092 <http://10.16.208.220:9092 > <http://10.16.208.220:9092>>, > 10.16.209.24:9092 > <http://10.16.209.24:9092 > <http://10.16.209.24:9092>>, > 10.16.209.146:9092 <http://10.16.209.146:9092 > <http://10.16.209.146:9092>>"* > *ZOOKEEPER="10.16.209.44:2181 <http://10.16.209.44:2181 > <http://10.16.209.44:2181>>"* > *declare -a topics=($(/opt/kafka/bin/kafka-topics.sh --zookeeper > ${ZOOKEEPER} --list | tr "\n" " "))* > *for topic in ${topics[@]}* > *do* > * messages=$(/opt/kafka/bin/kafka-run-class.sh > kafka.tools.GetOffsetShell --broker-list ${BROKERS} --topic $topic --time > -1| tr ":" " " | awk '{ sum += $3 } END { print sum }')* > * echo "$topic has $messages messages"* > *done* > > but I'd like to sample also the topic usage, and do a match between the 2 > lists. > Is there any JMX metric I can use to make sure nobody is receiving/creating > messages for each topic like a total counter or read/write per topic? > Cassandra has something like that so I was wondering if Kafka has something > like that as well. > Looking forward to your reply. > Regards > L. >