Re: Kafka java consumer processes duplicate messages

2016-08-02 Thread Amit K
l, Dave < > > > dave.tauz...@surescripts.com > > > > wrote: > > > > > > > If you kill a broker, then any uncommitted messages will be replayed. > > > > > > > > -Dave > > > >

Re: Kafka java consumer processes duplicate messages

2016-08-02 Thread R Krishna
> On Mon, Aug 1, 2016 at 11:35 AM, Tauzell, Dave < > > dave.tauz...@surescripts.com > > > wrote: > > > > > If you kill a broker, then any uncommitted messages will be replayed. > > > > > > -Dave > > >

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread Amit K
ages will be replayed. > > > > -Dave > > > > From: R Krishna > > Sent: Monday, August 1, 2016 1:32 PM > > To: users@kafka.apache.org > > Subject: Re: Kafka java consumer processes duplicate messages > > > > Remember readi

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread R Krishna
gt; -Dave > > From: R Krishna > Sent: Monday, August 1, 2016 1:32 PM > To: users@kafka.apache.org > Subject: Re: Kafka java consumer processes duplicate messages > > Remember reading about these options for higher consumer guarantees: >

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread Tauzell, Dave
If you kill a broker, then any uncommitted messages will be replayed. -Dave From: R Krishna Sent: Monday, August 1, 2016 1:32 PM To: users@kafka.apache.org Subject: Re: Kafka java consumer processes duplicate messages Remember reading about these options

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread R Krishna
Remember reading about these options for higher consumer guarantees: Unclean.leader.election = false Auto.offset.commit = falseconsumer side Commit after processingsyncCommit() regularly What about your producer, does it wait until it reaches all replicas in ISR, i.e., ack=all or none? Not

Kafka java consumer processes duplicate messages

2016-08-01 Thread Amit K
Hi, I am kind of new to Kafka. I have set up a 3 node kafka (1 broker per machine) cluster with 3 node zookeer cluster. I am using Kafka 0.9.0.0 version. The set up works fine wherein from my single producer I am pushing a JSON string to Kafka to a topic with 3 partitions and replication factor o