Hi I have a small proposal for improvement of Kafka that I would like to discuss with you. Kafka currently has rate metrics exposed per topic for the number of bytes ingested. I am interested in a volume metric.
I would like to have direct answer to the question how much data is ingested in a topic over a period of time. Disk usage does not suffice for the following reasons: - Disk allocation will be done in chunks which will be filled by ingested messages over time - The disk can be cleared based on retention period but this metric can potentially track volume ingestion over a period longer than the retention period. For example, I might have an 8 hour retention period but I might want to find out how much data was ingested on a topic in a day. Do you see any concerns over having such a metric? Regards, Amitav
