Re: 答复: [ANNOUNCE] New Committer: Dong Lin

Congrats Dong! On Wed, Mar 28, 2018 at 5:15 PM Dong Lin wrote: > Thanks everyone!! > > It is my great pleasure to be part of the Apache Kafka community and help > make Apache Kafka more useful to its users. I am super excited to be a > Kafka committer and I am hoping to contribute more to its de

Re: Higher CPU going from 0.8.1 to 0.8.2.1

Have you done a profiling on your broker process? Any hot code path differences between these two versions? Thanks, -Tao On Fri, Aug 21, 2015 at 3:59 PM, Rajiv Kurian wrote: > The only thing I notice in the logs which is a bit unsettling is about a > once a second rate of messages of the type >

Re: kafka-producer-perf-test.sh - No visible difference between request-num-acks 1 and -1

Hi Prabhjot, Do you intent to use the old producer performance microbenchmark? Thanks, -Tao On Fri, Aug 21, 2015 at 2:41 AM, Prabhjot Bharaj wrote: > Hi, > > I'm using Kafka 0.8.2.1 with the default zookeeper build that comes along > the bundle > > I have setup a 5 machine cluster and on the s

Re: thread that handle client request in Kafka brokers

ving parallelism in your pipe. > These include batching messages for bulk transfer, along with callbacks to > provide acknowledgement. Kafka obviously provides a great degree of > parallelism across brokers... > > Hope that helps. > > B > > > > > On 15 Aug 2015, a

Re: Reduce latency

kes sense if I set a fixed throughput. Why do I need to set a > fixed throughput for my test instead of just set the expected throughput to > be -1 (as much as possible)? > > Thanks. > > On Tue, Aug 18, 2015 at 2:43 PM, Tao Feng wrote: > > > Hi Yuheng, >

Re: Reduce latency

Hi Yuheng, The 1 record/s is just a param for producerperformance for your producer target tput. It only takes effect to do the throttling if you tries to send more than 1 record/s. The actual tput of the test depends on your producer config and your setup. -Tao On Tue, Aug 18, 2015 at

Re: Reduce latency

If you run producerPerformance test, there is only one thread per KafkaProducer doing the actual sending. But one network request could contain multiple batches from what I understand. On Mon, Aug 17, 2015 at 5:42 PM, Yuheng Du wrote: > Thank you Kishore, I made the buffer twice the size of the

thread that handle client request in Kafka brokers

Hi All, I was told that only single thread handle client producer request in Kafka broker which may potentially be a performance problem with request queue-up if we have many small requests. I am wondering which part of the code I should read to understand the above logic Thanks, -Tao

logging in Kafka new producer

Hi, I am wondering what Kafka new producer uses for logging? Is it log4j? Thanks, -Tao

logging for Kafka new producer

Hi, I am wondering what Kafka new producer uses for logging. Is it log4j? Thanks, -Tao

Re: kafka log flush questions

tiple redundant replicas to manage your durability concerns, if you can. > > B > > > On 7 Aug 2015, at 05:49, Tao Feng wrote: > > > > Hi , > > > > I am trying to understand the Kafka log flush behavior. My understanding > is > > when the broker spec

kafka log flush questions

Hi , I am trying to understand the Kafka log flush behavior. My understanding is when the broker specifies broker config param "log.flush.interval.ms", it will specify log config param "flush.ms" internally. In logManager logic, when the log exceed flush.ms, it will call Log.flush which will call

Kafka new producer thread safe question

Hi Kafka experts, I am trying to understand how Kafka new producer works. From the Kafka new producer code/comment, it indicates the producer is thread safe which we could have multiple clients per KafkaProducer. Is RecordAccumulator the only place to take care thread synchronization for KafkaProd

Re: Latency test

Sorry Yufeng, You should change it in $KAFKA_HEAP_OPTS. On Wed, Jul 15, 2015 at 3:09 PM, Tao Feng wrote: > Hi Yuheng, > > You could add the -Xmx1024m in > https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh > KAFKA_JVM_PERFORMANCE_OPTS. > > > > On We

Re: Latency test

ka-run-class.sh kafka.tools.TestEndToEndLatency 192.168.1.3:9092 > 192.168.1.1:2181 speedx3 5000 100 1 -Xmx1024m > > It promped that it is not correct. So where should I put the -Xmx1024m > option? Thanks. > > > > On Wed, Jul 15, 2015 at 3:44 AM, Tao Feng wrote: > >

Re: Latency test

is outdated already. The error message shows: > > USAGE: java kafka.tools.TestEndToEndLatency$ broker_list zookeeper_connect > topic num_messages consumer_fetch_max_wait producer_acks > > > Can anyone helps me what should be put in consumer_fetch_max_wait? Thanks. > > On Tue, Jul

Re: Latency test

I think ProducerPerformance microbenchmark only measure between client to brokers(producer to brokers) and provide latency information. On Tue, Jul 14, 2015 at 11:05 AM, Yuheng Du wrote: > Currently, the latency test from kafka test the end to end latency between > producers and consumers. > >