[
https://issues.apache.org/jira/browse/KAFKA-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar resolved KAFKA-1436.
------------------------------
Resolution: Duplicate
> Idempotent Producer / Duplicate Detection
> -----------------------------------------
>
> Key: KAFKA-1436
> URL: https://issues.apache.org/jira/browse/KAFKA-1436
> Project: Kafka
> Issue Type: Improvement
> Components: consumer, producer
> Affects Versions: 0.8.0, 0.8.1, 0.8.1.1, 0.8.2.0, 0.10.1.0
> Reporter: James Thornton
> Assignee: Neha Narkhede
>
> Dealing with duplicate messages is one of the major issues for teams using
> Kafka, and Jay Kreps posted a page on implementing an Idempotent Producer to
> address this issue:
> https://cwiki.apache.org/confluence/display/KAFKA/Idempotent+Producer
> MapDB 1.0 (https://github.com/jankotek/MapDB) was just released, and either
> it or Java Chronicle (https://github.com/OpenHFT/Java-Chronicle/) could be
> embedded within each broker to provide a high-performance, random-access,
> off-heap store for request IDs.
> As Jay points out in his post, global unique request IDs probably aren't
> needed, but if that need should arise, Twitter's Snowflake service
> (https://github.com/twitter/snowflake/) might be useful.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
