Hi. Looking for some insights here. We use Kafka at a large scale, and have lots of microservices using Kafka for all sorts of things. Our biggest challenge nowadays is to track which topics are used and which are not. I have considered looking at consumer groups to identify which applications consume from each topic, but then I realized we have lots of microservices that do not use consumer groups (lots of goka views on top of Kafka topics).
Is there any architecture solution that can help track consumers of a Kafka topic? Thought that I could use ACL to restrict the creation of Kafka Topics and use a gatekeeper service to centralize and log who creates the topics, but still that does not block anyone from consuming from them. Any thoughts on that? Thanks Murilo
