You can achieve exactly once on a consumer by enabling read committed and manually committing the offset as soon as you receive a message. That way you know that at next poll you won't get old message again.
On Fri, Sep 27, 2019, 6:24 AM christopher palm <cpa...@gmail.com> wrote: > I had a similar question, and just watched the video on the confluent.io > site about this. > From what I understand idempotence and transactions are there to solve the > duplicate writes and exactly once processing, respectively. > > Is what you are stating below is that this only works if we produce into a > kafka topic and consume from it via a kafka stream, but a regular > kafka consumer won't get the guarantee of exactly once processing? > > Thanks, > Chris > > > On Sat, Aug 31, 2019 at 12:29 AM Matthias J. Sax <matth...@confluent.io> > wrote: > > > Exactly-once on the producer will only ensure that no duplicate writes > > happen. If a downstream consumer fails, you might still read message > > multiple times for all cases (ie, without idempotence, with idempotence > > enabled, or if you use transactions). > > > > Note, that exactly-once is designed for a read-process-write pattern, > > but not for a write-read pattern. > > > > -Matthias > > > > > > > > On 8/30/19 1:00 PM, Peter Groesbeck wrote: > > > For a producer that emits messages to a single topic (i.e. no single > > > message is sent to multiple topics), will enabling idempotency but not > > > transactions provide exactly once guarantees for downstream consumers > of > > > said topic? > > > > > > Ordering is not important I just want to make sure consumers only > > consumer > > > messages sent once. > > > > > > > >
