Your understanding sounds correct.
One follow up: even idempotent producer itself, gives you
"exactly-once", because for many use cases it's important to not write
duplicates into a topic.
Thus, I would not say that you need transactions to do exactly-once (but
I guess it depends what your exact
Thanks Matthias for the answers and the update to FAQ. I understand
exactly-once semantics much better now.
In summary, producer side idempotence can be used on its own using
enable.idempotence parameter (which underneath uses PID and sequence number
combo). However, if exactly-once semantics is n
Done. Feel free to extend/correct/complete etc.
-Matthias
On 2/20/19 9:56 AM, Guozhang Wang wrote:
> Since we've seen quite a lot of questions recently about EOS on the
> mailing list. I think it worth adding an FAQ entry here:
>
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ
>
> So th
Since we've seen quite a lot of questions recently about EOS on the mailing
list. I think it worth adding an FAQ entry here:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ
So that we can refer future questions to the page than answering them
repeatedly. @Matthias J Sax : would you like to
Even if the question was sent 4 times to the mailing list, I am only
answering is exactly-once (sorry for the bad joke -- could not resist...)
You have to distinguish between "idempotent producer" and "transactional
producer".
If you enable idempotent writes (config `enable.idempotence`), your
p
Hi,
Our data getting into Kafka is transactional in nature and hence I am
trying to understand EOS better. My present understanding is as below:
It is mentioned that when producer starts, it will have a new PID, but only
valid till the session. Does that mean, is it a pre-requisite to have the
sa
