Version: Kafka 2.0.0 (Believe the problem also exists on 2.2.0) The producer thread in MirrorMaker will go repeatedly sending loop without any backoffs when encountered metadata errors. For example, the following error messages will repeat itself for hundred times with very small interval (100ms) when it encountered some metadata error.
The producer thread in mirror maker was configured with 'retries=Integer.MAX', so this resending is very annoying. By looking at Sender.java, the few other places when we call metadata.requestUpdate(), it is usually after time.sleep(retryBackoffMs), but not at this particular place (in method completeBatch()) [2019-03-11 02:46:45,988] WARN [Producer clientId=producer-1] Received invalid metadata error in produce request on partition incomplete_ad_conversion_secondary-32 due to org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.. Going to request metadata update now (org.apache.kafka.clients.producer.internals.Sender) [2019-03-11 02:46:46,094] WARN [Producer clientId=producer-1] Got error produce response with correlation id 1488890 on topic-partition incomplete_ad_conversion_secondary-32, retrying (2147482734 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender) [2019-03-11 02:46:46,094] WARN [Producer clientId=producer-1] Received invalid metadata error in produce request on partition incomplete_ad_conversion_secondary-32 due to org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.. Going to request metadata update now (org.apache.kafka.clients.producer.internals.Sender)
