Hi, That's good, but somewhat mysterious :) I'd like to help accelerate exactly-one behaviour. Perhaps it might be a candidate for the release after 0.10, if there's sufficient interest in the community.
Could someone share the latest thinking on this subject? I'm not sure what the appropriate forum is. How have you previously been sharing high-level architectural plans for people to pick up and deliver? Since there's no KIP I guess it's a bit premature to discuss on the KIP call. Thanks Andrew > Subject: Re: Exactly-once publication behaviour > From: b...@confluent.io > Date: Fri, 19 Feb 2016 15:13:07 -0800 > To: users@kafka.apache.org > > Hi Andrew > > There are plans to add exactly once behaviour. This will likely be a little > more than Idempotent producers with the motivation being to provide better > delivery guarantees for Connect, Streams and Mirror Maker. > > B > > > >> On 19 Feb 2016, at 13:54, Andrew Schofield wrote: >> >> When publishing messages to Kafka, you make a choice between at-most-once >> and at-least-once delivery, depending on whether you wait for >> acknowledgments and whether you retry on failures. In most cases, those >> options are good enough. However, some systems offer exactly-once >> reliability too. Although my view is that the practical use of exactly-once >> is limited in the situations that Kafka is generally used for, when you're >> connecting other systems to Kafka or bridging between protocols, I think >> there is value in propagating the reliability level that the other system >> expects. >> >> As a consumer, you can manage your offset and get exactly-once delivery, or >> more likely exactly-once processing, of the messages. >> >> I've read about idempotent producers >> (https://cwiki.apache.org/confluence/display/KAFKA/Idempotent+Producer) and >> I know there's been some discussion about transactions too. >> >> Is there a plan to provide the tools to enable exactly-once publication >> behaviour? Is this a planned enhancement to Kafka Connect? Is there already >> some technique that people are using effectively to get exactly-once? >> >> Andrew Schofield >
