Hello,
I've been trying to monitor a Kafka broker using the CSVMetricsReporter.
However, when I start the broker, only a few csv files are created in the
directory, and then there are repeated IOExceptions in the kafka.out log
stating that CsvReporter.createStreamForMetric cannot create the file
{path_to_metrics_dir}/LocalTimeMs.csv
I can recreate this problem in 0.8.2.1, 0.8.2.2, and 0.9.0.0. CSV reporting
works as expected in 0.8.1.1.
I believe the problem is that in the later versions CSV files are being
named from only the metric name, and not the tags. In 0.8.1.1, the
LocalTimeMs metrics are given a name prefixed with a distinguishing source,
while in later versions this information is passed as a tag. These tags are
passed into the MetricName constructor in
KafkaMetricsGroup.explicitMetricName, but as the mBeanName and not as the
name.
This is causing a name conflict, since CSV files are named off of the name
field, not the mBeanName.
For a similar reason, I cannot get metrics on partition specific data. The
CSV reporter seems to not try any other metrics after raising an exception
on LocalTimeMs. All metrics are published correctly to JMX.
Has anyone encountered this problem? Is there a way to make CSV metrics
work in the stable release (0.8.2.2)?
It is possible I have missed something in following the code, as I am not
very literate in Scala, but I am reasonably confident that my understanding
of the code lines up with the behavior I've seen.
I believe this behavior is related to an old issue, KAFKA-542
<https://issues.apache.org/jira/browse/KAFKA-542>
Best,
Andrew Grasso
