Re: Kafka java consumer processes duplicate messages

2016-08-02 Thread Amit K
l, Dave < > > > dave.tauz...@surescripts.com > > > > wrote: > > > > > > > If you kill a broker, then any uncommitted messages will be replayed. > > > > > > > > -Dave > > > >

Re: Kafka java consumer processes duplicate messages

2016-08-02 Thread R Krishna
> On Mon, Aug 1, 2016 at 11:35 AM, Tauzell, Dave < > > dave.tauz...@surescripts.com > > > wrote: > > > > > If you kill a broker, then any uncommitted messages will be replayed. > > > > > > -Dave > > >

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread Amit K
ages will be replayed. > > > > -Dave > > > > From: R Krishna > > Sent: Monday, August 1, 2016 1:32 PM > > To: users@kafka.apache.org > > Subject: Re: Kafka java consumer processes duplicate messages > > > > Remember readi

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread R Krishna
gt; -Dave > > From: R Krishna > Sent: Monday, August 1, 2016 1:32 PM > To: users@kafka.apache.org > Subject: Re: Kafka java consumer processes duplicate messages > > Remember reading about these options for higher consumer guarantees: >

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread Tauzell, Dave
If you kill a broker, then any uncommitted messages will be replayed. -Dave From: R Krishna Sent: Monday, August 1, 2016 1:32 PM To: users@kafka.apache.org Subject: Re: Kafka java consumer processes duplicate messages Remember reading about these options

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread R Krishna
Remember reading about these options for higher consumer guarantees: Unclean.leader.election = false Auto.offset.commit = falseconsumer side Commit after processingsyncCommit() regularly What about your producer, does it wait until it reaches all replicas in ISR, i.e., ack=all or none? Not

Kafka java consumer processes duplicate messages

2016-08-01 Thread Amit K
Hi, I am kind of new to Kafka. I have set up a 3 node kafka (1 broker per machine) cluster with 3 node zookeer cluster. I am using Kafka 0.9.0.0 version. The set up works fine wherein from my single producer I am pushing a JSON string to Kafka to a topic with 3 partitions and replication factor o

kafka java consumer not consuming messsages produced by remote client

2015-11-23 Thread Kudumula, Surender
line consumer works but my kafka java consumer doesn't consume messages produced by remote client but when I produce messages from command line in my local machine where kafka client is subscribed to the same topic it works and consumes all the messages. bin/kafka-console-consumer.sh --zoo

Re: Kafka java consumer

2015-08-14 Thread Ewen Cheslack-Postava
There's not a precise date for the release, ~1.5 or 2 months from now. On Fri, Aug 14, 2015 at 3:45 PM, Abhijith Prabhakar wrote: > Thanks Ewen. Any idea when we can expect 0.8.3? > > > > On Aug 14, 2015, at 5:36 PM, Ewen Cheslack-Postava > wrote: > > > > Hi Abhijith, > > > > You should be usi

Re: Kafka java consumer

2015-08-14 Thread Abhijith Prabhakar
Thanks Ewen. Any idea when we can expect 0.8.3? > On Aug 14, 2015, at 5:36 PM, Ewen Cheslack-Postava wrote: > > Hi Abhijith, > > You should be using KafkaProducer, but KafkaConsumer is not ready yet. The > APIs are included in 0.8.2.1, but the implementation is not ready. Until > 0.8.3 is rel

Re: Kafka java consumer

2015-08-14 Thread Ewen Cheslack-Postava
Hi Abhijith, You should be using KafkaProducer, but KafkaConsumer is not ready yet. The APIs are included in 0.8.2.1, but the implementation is not ready. Until 0.8.3 is released, you cannot rely only on kafka-clients if you want to write a consumer. You'll need to depend on the main kafka jar and

Kafka java consumer

2015-08-14 Thread Abhijith Prabhakar
Hi All, I am newbie to Kafka and was looking to use java client implementation org.apache.kafka:kafka-clients:0.8.2.1. I was trying to write a consumer group using example given here: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example