Hi Edwin, 1. Yes. 2. Yes. 3. Yes; and there is no difference in terms of parallelism.
In the new consumer client (org.apache.kafka.clients.consumer.KafkaConsumer) that is going to be out in the next release, each consumer instance is single-threaded, i.e. it will only have one thread for fetching data. On Wed, Feb 4, 2015 at 3:43 AM, Zijing Guo <alter...@yahoo.com.invalid> wrote: > Hi,I have some question regarding how kafka consumers achieve parallel > consuming for one topic. Say I have 2 partitions for topic1 and I have a > consumer Group A, now:1: If no consumer under consumer Group A subscribe > topic1, then no message will be delivery to this consumer group.2: If there > is only 1 consumer under consumer Group A that subscribe topic1, then this > consumer will consume all the data from topic1 that among the 2 > partitions?3:in the High level consumer example > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Exampleit > says: "if you provide more threads than there are partitions on the topic, > some threads will never see a message > - if you have more partitions than you have threads, some threads will > receive data from multiple partitions" > so back to my example, If I create 2 threads under one consumer instance > that I created and will each thread corresponding to a specific partition > under the topic1? what is the difference that I create 2 consumers, each > with only 1 thread under same consumer group and consume topic1? Is kafka > parallel consuming really down to the thread level that with 1 consumer > instance or multiple consumer instance, each with 1 thread? > ThanksEdwin -- -- Guozhang
