I've got a sandbox cluster (without any control on topic creation) full of unused topics and I'd like to cleanup all the those topics. I was able to identify all the topics with 0 messages with the following lines:
*#!/bin/bash* *set -e* *BROKERS="10.16.208.220:9092 <http://10.16.208.220:9092>,10.16.209.24:9092 <http://10.16.209.24:9092>,10.16.209.146:9092 <http://10.16.209.146:9092>"* *ZOOKEEPER="10.16.209.44:2181 <http://10.16.209.44:2181>"* *declare -a topics=($(/opt/kafka/bin/kafka-topics.sh --zookeeper ${ZOOKEEPER} --list | tr "\n" " "))* *for topic in ${topics[@]}* *do* * messages=$(/opt/kafka/bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list ${BROKERS} --topic $topic --time -1| tr ":" " " | awk '{ sum += $3 } END { print sum }')* * echo "$topic has $messages messages"* *done* but I'd like to sample also the topic usage, and do a match between the 2 lists. Is there any JMX metric I can use to make sure nobody is receiving/creating messages for each topic like a total counter or read/write per topic? Cassandra has something like that so I was wondering if Kafka has something like that as well. Looking forward to your reply. Regards L.
