The new producer uses a different built-in metrics package. Currently, it only supports a jmx reporter for the metrics. So you will have to get the metrics from jmx.
We can add the csv reporter in ProducerPerformance for the new producer by using the new metrics api. Could you file a jira for that? Thanks, Jun On Wed, Feb 4, 2015 at 11:48 PM, Xinyi Su <xiny...@gmail.com> wrote: > Hi, > > You are right. The file is created by Kafka and Kafka producer csv reporter > append metrics continuously during perf test. It should not be created > continuously during test but create once only. > > Another thing is when I turn on "--new-producer" and "--sync" option, > nothing is appended into ProducerRequestSize.csv. From source code, I > have not got any NewShinyProducer record stats code. It may be the reason. > > Thanks. > Xinyi > > On 5 February 2015 at 15:26, Jaikiran Pai <jai.forums2...@gmail.com> > wrote: > > > >> java.io.IOException: Unable to create /tmp/PerfTopic22_1/ > > ProducerRequestSize.csv > > > > It looks like a file with that exact same name already exists which is > > causing that file creation request to fail. This indicates that probably > > the metric name (ProducerRequestSize) from which the file is created, is > > duplicate for whatever reason. > > > > -Jaikiran > > > > > > On Thursday 05 February 2015 12:49 PM, Xinyi Su wrote: > > > >> Hi, > >> > >> I need to get more metrics from csv reporter. If turn off csv-reporter, > >> few output is shown. > >> > >> Thanks. > >> Xinyi > >> > >> On 5 February 2015 at 13:09, tao xiao <xiaotao...@gmail.com> wrote: > >> > >> Hi, > >>> > >>> In order to get it work you can turn off csv-reporter. > >>> > >>> On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xiny...@gmail.com> wrote: > >>> > >>> Hi, > >>>> > >>>> Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run kafka > >>>> producer performance test. > >>>> > >>>> The test cannot continue because of some exceptions thrown which does > >>>> not > >>>> occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar > which > >>>> > >>> is > >>> > >>>> the latest version on maven repository. > >>>> > >>>> -bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker > >>>> list> > >>>> --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000 > >>>> --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1 > >>>> --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1 > >>>> > >>>> java.io.IOException: Unable to create > >>>> /tmp/PerfTopic22_1/ProducerRequestSize.csv > >>>> at > >>>> > >>>> > >>>> com.yammer.metrics.reporting.CsvReporter.createStreamForMetric( > >>> CsvReporter.java:141) > >>> > >>>> at > >>>> > >>>> > >>>> com.yammer.metrics.reporting.CsvReporter.getPrintStream( > >>> CsvReporter.java:257) > >>> > >>>> at > >>>> > >>> > com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22) > >>> > >>>> at > >>>> > >>>> com.yammer.metrics.reporting.CsvReporter$1.getStream( > >>> CsvReporter.java:156) > >>> > >>>> at > >>>> > >>>> > >>>> com.yammer.metrics.reporting.CsvReporter.processHistogram( > >>> CsvReporter.java:194) > >>> > >>>> at > >>>> > >>>> > >>>> com.yammer.metrics.reporting.CsvReporter.processHistogram( > >>> CsvReporter.java:22) > >>> > >>>> at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231) > >>>> at com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163) > >>>> at > >>>> > >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >>> > >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > >>>> at > >>>> > >>>> > >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ > >>> ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > >>> > >>>> at > >>>> > >>>> > >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ > >>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > >>> > >>>> at > >>>> > >>>> > >>>> java.util.concurrent.ThreadPoolExecutor.runWorker( > >>> ThreadPoolExecutor.java:1145) > >>> > >>>> at > >>>> > >>>> > >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run( > >>> ThreadPoolExecutor.java:615) > >>> > >>>> at java.lang.Thread.run(Thread.java:745) > >>>> > >>>> > >>> > >>> -- > >>> Regards, > >>> Tao > >>> > >>> > > >
