Yuheng, >From the command you gave it looks like you are configuring the perf test to send data as fast as possible (the -1 for target throughput). This means it will always queue up a bunch of unsent data until the buffer is exhausted and then block. The larger the buffer, the bigger the queue. This is where the latency comes from. This is exactly what you would expect and what the buffering is supposed to do.
If you want to measure latency this test doesn't really make sense, you need to measure with some fixed throughput. Instead of -1 enter the target throughput you want to measure latency at (e.g. 100000 records/sec). -Jay On Thu, Aug 13, 2015 at 12:18 PM, Yuheng Du <yuheng.du.h...@gmail.com> wrote: > Thank you Alvaro, > > How to use sync producers? I am running the standard ProducerPerformance > test from kafka to measure the latency of each message to send from > producer to broker only. > The command is like "bin/kafka-run-class.sh > org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 -1 > acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 > buffer.memory=67108864 batch.size=8196" > > For running producers, where should I put the producer.type=sync > configuration into? The config/server.properties? Also Does this mean we > are using batch size of 1? Which version of Kafka are you using? > thanks. > > On Thu, Aug 13, 2015 at 3:01 PM, Alvaro Gareppe <agare...@gmail.com> > wrote: > > > Are you measuring latency as time between producer and consumer ? > > > > In that case, the ack shouldn't affect the latency, cause even tough your > > producer is not going to wait for the ack, the consumer will only get the > > message after its commited in the server. > > > > About latency my best result occur with sync producers, but the > throughput > > is much lower in that case. > > > > About not flushing to disk I'm pretty sure that it's not an option in > kafka > > (correct me if I'm wrong) > > > > Regards, > > Alvaro Gareppe > > > > On Thu, Aug 13, 2015 at 12:59 PM, Yuheng Du <yuheng.du.h...@gmail.com> > > wrote: > > > > > Also, the latency results show no major difference when using ack=0 or > > > ack=1. Why is that? > > > > > > On Thu, Aug 13, 2015 at 11:51 AM, Yuheng Du <yuheng.du.h...@gmail.com> > > > wrote: > > > > > > > I am running an experiment where 92 producers is publishing data > into 6 > > > > brokers and 10 consumer are reading online data simultaneously. > > > > > > > > How should I do to reduce the latency? Currently when I run the > > producer > > > > performance test the average latency is around 10s. > > > > > > > > Should I disable log.flush? How to do that? Thanks. > > > > > > > > > > > > > > > -- > > Ing. Alvaro Gareppe > > agare...@gmail.com > > >
