Hi, I guess message production duplicate scenario in Kafka is, when a producer commits the data but does not get an ack (broker or network fails AFTER commit) and retries.
I got thinking that the retry can be caught by the broker which could then identify the previous message with a unique message ID before committing them again but it requires a small lookup table for the previous commit ID for each producer. Can I please know what is wrong about this picture, since Kafka does not do this? Thanks Arun
