Hi Erik, What do you mean by "this must be done external to " ?
If the cluster still uses ZK for metadata, you can construct a read-only Zookeeper client to get zNode creation timestamp for the topic from ZK node, which reflects the creation time. Also determining unused topics can be tricky if consumers come and go by irregular basis, for example some batch jobs for BI and/or monthly generated reports, etc. Check this blog post to see how LinkedIn does it. https://engineering.linkedin.com/content/engineering/en-us/blog/2022/topicgc_how-linkedin-cleans-up-unused-metadata-for-its-kafka-clu IMHO hard deletes should be avoided for any kind of data platform. Kind regards, OSB On Sun, Oct 20, 2024, 14:24 Erik Godding Boye <egb...@gmail.com> wrote: > Hi Kafka experts, > > I wrote a small utility application to terminate (delete) unused topics in > our clusters. The application is hosted at > https://github.com/statnett/k3a-topic-terminator (but this is not an > attempt to promote the app; it's fresh ;-). We just got rid of 2816 topics > in one of our clusters, and now we want to find even more garbage. > > So my question is: *Can the topic creation timestamp be obtained through a > Kafka Admin Client (or Client)?* > > I think the answer is NO, but would it be possible to implement such a > feature? As a user, I would expect to find this information through the > TopicDescription > < > https://kafka.apache.org/38/javadoc/org/apache/kafka/clients/admin/TopicDescription.html > > > class.As far as I understand, this information is available in the cluster > - either within Zookeepers and/or on the __cluster_metadata topic. I have > tried to set up a consumer towards the metadata topic, but no luck so far. > I am very interested in potential workarounds until this feature is > eventually implemented! In my case, this *must* be done external to the > brokers/controllers/zookeepers. Running a shell script or CLI tool is not a > feasible option (in this case). > > Regards. > Erik >
