Hi Yong Gang, Idempotence is not guaranteed if you increase max in flight requests. Because if 5 requests are dispatched, but 2 requests fail and are retried, the messages in those 2 requests will arrive after the messages in the other three requests, so ordering will be changed.
Assuming everything is happy in your cluster, network etc., you won't see immediate negative effects on idempotence when increasing that property. But when the cluster/network/etc. is heavily loaded, then you'll start to see the negative effects. Does that make sense? Kind regards, Liam Clarke-Hutchinson On Fri, 23 Apr. 2021, 11:44 am Yong Gang, <che.yongg...@gmail.com> wrote: > Hi guys, > > We are using Kafka Connect to do Kafka to Kafka data replication between > DCs. > For this config item *producer.max.in.flight.requests.per.connection* the > default value in KafkaProducer (Kafka client) is 5 and can still > maintain idempotence, but in Kafka Connect the default value of this is 1. > > I wonder if increasing *producer.max.in.flight.requests.per.connection *to > 5 can Kafka Connect still be idempotent? > > Thanks, > YongGang >
