According to code of org.apache.kafka.clients.producer.KafkaProducer. It has 1 ioThread
private final Thread <http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/Thread.java#Thread> ioThread; String <http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/String.java#String> ioThreadName = "kafka-producer-network-thread" + (clientId.length <http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/String.java#String.length%28%29>() > 0 ? " | " + clientId : ""); 210 <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#210> <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#> this.ioThread <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#KafkaProducer.0ioThread> = new KafkaThread <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/common/utils/KafkaThread.java#KafkaThread>(ioThreadName, this.sender <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#KafkaProducer.0sender>, true); 211 <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#211> <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#> this.ioThread <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#KafkaProducer.0ioThread>.start <http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/Thread.java#Thread.start%28%29>(); 212 <http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#212> is ioThread is the one responsible for sending message from inmemory buffer to kafka broker . So where is SyncProducer got created and is SyncProducer responsible for sending message from inmemory buffer to kafka broker and where is that got created? On Tue, Jun 30, 2015 at 9:16 PM, Guozhang Wang <wangg...@gmail.com> wrote: > One ProducerSendThread is always the case for producer, but inside the > thread there is one SyncProducer for each destination broker. I think that > blog may mis-understand the design a bit. > > Guozhang > > On Tue, Jun 30, 2015 at 1:45 AM, Shushant Arora <shushantaror...@gmail.com > > > wrote: > > > According to > > https://engineering.gnip.com/kafka-async-producer/ > > There is one ProducerSendThread per broker. Is it changed from 0.8.2 ? > to > > use single send thread irrespective of broker or partitions. > > > > > > On Mon, Jun 29, 2015 at 11:27 PM, Guozhang Wang <wangg...@gmail.com> > > wrote: > > > > > Hi Shushant, > > > > > > The number of ProducerSendThread will be always one, regardless of the > > > number of destination brokers, or the number of partitions. > > > > > > Guozhang > > > > > > On Mon, Jun 29, 2015 at 9:38 AM, Shushant Arora < > > shushantaror...@gmail.com > > > > > > > wrote: > > > > > > > Hi > > > > > > > > Does kafka async producer creates thread(ProducerSendThread) in > > producer > > > > memory based on no number of partitions or brokers in kafka cluster > to > > > > which it will write. > > > > > > > > If my cluster had 1000 partitions does each producer will have 1000 > > > threads > > > > running always ? > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > -- > -- Guozhang >
