Hey De Gao, The controller is the one that always elects a new leader. When that happens that metadata is changed on the controller and once committed it’s broadcast to all brokers in the cluster. In KRaft this would be via a PartitonChange record that each broker will fetch from the controller. In ZK it’d be via an RPC from the controller to the broker.
In either case each broker might get the notification at a different time. No ordering guarantee among the brokers. But eventually they’ll all know the new leader which means eventually the Produce will fail with NotLeader and the client will refresh its metadata and find out the new one. In between all that leadership movement, there are various ways messages can get duplicated or lost. However if you use the idempotent producer I believe you actually won’t see dupes or missing messages so if that’s an important requirement you could look into that. The producer is designed to retry in general and when you use the idempotent producer some extra metadata is sent around to dedupe any messages server-side that were sent multiple times by the client. If you’re interested in learning more Kafka internals I highly recommend this blog series https://www.confluent.io/blog/apache-kafka-architecture-and-internals-by-jun-rao/ Hope that helped a bit. Andy Sent from my iPhone > On Nov 20, 2023, at 2:07 AM, De Gao <d...@live.co.uk> wrote: > > Hi all I have a interesting question here. > > Let's say we have 2 broker B1 B2, controller C and producer P1, P2...Pn. > Currently B1 holds the partition leader and Px is constantly producing > messages to B1. We want to move the partition leadership to B2. How does the > leadership change synced between B1, B2, C, and Px that it is guaranteed that > all the parties acknowledged the leadership change in the right order? Was > there a break of produce flow in between? Any chance of message lost? > > Thanks > > De Gao
