RE: Producer duplicates

2015-02-17 Thread Arunkumar Srambikkal (asrambik)
Thanks Jay. Any idea, when this feature can be expected? Are there any patches for it? Thanks Arun -Original Message- From: Jay Kreps [mailto:jay.kr...@gmail.com] Sent: Tuesday, February 17, 2015 10:16 PM To: users@kafka.apache.org Subject: Re: Producer duplicates There are some

Re: Producer duplicates

2015-02-17 Thread Jay Kreps
There are some details to get this right, the lookup table has to survive failures. But yes this is exactly what we would like to add: https://cwiki.apache.org/confluence/display/KAFKA/Idempotent+Producer -Jay On Tue, Feb 17, 2015 at 12:44 AM, Arunkumar Srambikkal (asrambik) < asram...@cisco.com

Producer duplicates

2015-02-17 Thread Arunkumar Srambikkal (asrambik)
Hi, I guess message production duplicate scenario in Kafka is, when a producer commits the data but does not get an ack (broker or network fails AFTER commit) and retries. I got thinking that the retry can be caught by the broker which could then identify the previous message with a unique mes