Hey Raffaele, the producer id is getting assigned upon receiving the producer.initTransaction call at the broker side. It guarantees the uniqueness of a producer for current lifecycle, which you don't have to configure manually.
Transactional API on the other hand, includes idempotent produce for sure. You need to provide a unique transactional.id though. Feel free to learn more high level stuff through this blog: https://www.confluent.io/blog/transactions-apache-kafka/. Boyang Boyang On Fri, May 15, 2020 at 11:48 AM Raffaele Esposito <rafaelral...@gmail.com> wrote: > In relation to producer idempotency Kafka documentation says: > > > ... Since each new instance of a producer is assigned a new, unique, PID, > we can only guarantee idempotent production within a single producer > session. > > > Does it mean that when we build a producer sourcing the data from whatever > source (MySql for example) we need to take care explicitly of this problem > > and put in place effective counter measures, like checking what was the > last message actually produced successfully on Kafka ? > > > What if we use transactions, even in the case we don't need really need > transactional logic on writing messages to different partitions in an all > or nothing fashion ? > > Would using transactions in our producer get rid of the idempotency issue ? > > > > Thanks! >
