ProducerSendThread and SyncProducer all come from the old Scala producer, I thought you meant that producer not the new Java producer?
Guozhang On Tue, Jun 30, 2015 at 9:09 AM, Shushant Arora <shushantaror...@gmail.com> wrote: > According to code of org.apache.kafka.clients.producer.KafkaProducer. > > It has 1 ioThread > > private final Thread > < > http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/Thread.java#Thread > > > ioThread; > > String < > http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/String.java#String > > > ioThreadName = "kafka-producer-network-thread" + (clientId.length > < > http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/String.java#String.length%28%29 > >() > > 0 ? " | " + clientId : ""); > > 210 < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#210 > > > > < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java# > > > > this.ioThread > < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#KafkaProducer.0ioThread > > > = new KafkaThread > < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/common/utils/KafkaThread.java#KafkaThread > >(ioThreadName, > this.sender < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#KafkaProducer.0sender > >, > true); > > 211 < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#211 > > > > < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java# > > > > this.ioThread > < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#KafkaProducer.0ioThread > >.start > < > http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/Thread.java#Thread.start%28%29 > >(); > > 212 < > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.kafka/kafka-clients/0.8.2.0/org/apache/kafka/clients/producer/KafkaProducer.java#212 > > > > > is ioThread is the one responsible for sending message from inmemory buffer > to kafka broker . So where is SyncProducer got created and is SyncProducer > responsible for sending message from inmemory buffer to kafka broker and > where is that got created? > > > > > > > On Tue, Jun 30, 2015 at 9:16 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > One ProducerSendThread is always the case for producer, but inside the > > thread there is one SyncProducer for each destination broker. I think > that > > blog may mis-understand the design a bit. > > > > Guozhang > > > > On Tue, Jun 30, 2015 at 1:45 AM, Shushant Arora < > shushantaror...@gmail.com > > > > > wrote: > > > > > According to > > > https://engineering.gnip.com/kafka-async-producer/ > > > There is one ProducerSendThread per broker. Is it changed from 0.8.2 ? > > to > > > use single send thread irrespective of broker or partitions. > > > > > > > > > On Mon, Jun 29, 2015 at 11:27 PM, Guozhang Wang <wangg...@gmail.com> > > > wrote: > > > > > > > Hi Shushant, > > > > > > > > The number of ProducerSendThread will be always one, regardless of > the > > > > number of destination brokers, or the number of partitions. > > > > > > > > Guozhang > > > > > > > > On Mon, Jun 29, 2015 at 9:38 AM, Shushant Arora < > > > shushantaror...@gmail.com > > > > > > > > > wrote: > > > > > > > > > Hi > > > > > > > > > > Does kafka async producer creates thread(ProducerSendThread) in > > > producer > > > > > memory based on no number of partitions or brokers in kafka cluster > > to > > > > > which it will write. > > > > > > > > > > If my cluster had 1000 partitions does each producer will have 1000 > > > > threads > > > > > running always ? > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
