When publishing messages to Kafka, you make a choice between at-most-once and at-least-once delivery, depending on whether you wait for acknowledgments and whether you retry on failures. In most cases, those options are good enough. However, some systems offer exactly-once reliability too. Although my view is that the practical use of exactly-once is limited in the situations that Kafka is generally used for, when you're connecting other systems to Kafka or bridging between protocols, I think there is value in propagating the reliability level that the other system expects.
As a consumer, you can manage your offset and get exactly-once delivery, or more likely exactly-once processing, of the messages. I've read about idempotent producers (https://cwiki.apache.org/confluence/display/KAFKA/Idempotent+Producer) and I know there's been some discussion about transactions too. Is there a plan to provide the tools to enable exactly-once publication behaviour? Is this a planned enhancement to Kafka Connect? Is there already some technique that people are using effectively to get exactly-once? Andrew Schofield
