Re: consuming only half the messages produced

2013-05-02 Thread Jun Rao
be carefully reading the following, next: > https://cwiki.apache.org/KAFKA/kafka-replication.html. > > thanks, > rob > > > -Original Message- > > From: Rob Withers [mailto:reefed...@gmail.com] > > Sent: Thursday, May 02, 2013 7:20 AM > > To: 'users@k

RE: consuming only half the messages produced

2013-05-02 Thread Rob Withers
; Sent: Thursday, May 02, 2013 7:20 AM > To: 'users@kafka.apache.org' > Subject: RE: consuming only half the messages produced > > Yes, I mean we can only consume half the messages produced. I followed the > high-level consumer example here: > https://cwiki.apache.org/c

RE: consuming only half the messages produced

2013-05-02 Thread Rob Withers
-- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Wednesday, May 01, 2013 11:15 PM > To: users@kafka.apache.org > Subject: Re: consuming only half the messages produced > > Partition is different from replicas. A topic can have one or more partitions > and each partition ca

Re: consuming only half the messages produced

2013-05-01 Thread Jun Rao
Partition is different from replicas. A topic can have one or more partitions and each partition can have one or more replicas. A consumer consumes data at partition level. In other words, a consumer gets the same data no matter how many replicas are there. When you say the consumer only gets half

consuming only half the messages produced

2013-05-01 Thread Rob Withers
Running a consumer group (createStreams()), pointing to the zookeeper and with the topic and 1 consumer thread, results in only half the messages being consumed. The topic was auto-created, with a replication factor of 2, but the producer was configured to produce to 2 brokers and so 4 partitions