Ok, great, that makes sense, Igor. Thanks. +1 (binding) on the KIP from me.
Ron > On Sep 13, 2023, at 11:58 AM, Igor Soarez <soa...@apple.com.invalid> wrote: > > Hi Ron, > > Thanks for drilling down on this. I think the KIP isn't really clear here, > and the metadata caching section you quoted needs clarification. > > The "hosting broker's latest registration" refers to the previous, > not the current registration. The registrations are only compared by > the controller, when handling the broker registration request. > > Suppose broker b1 hosts two partitions, t-1 and t-2, in two > directories, d1 and d2. The broker is registered, and the > metadata correlates the replicas to their respective directories. > i.e. OnlineLogDirs=[d1,d2] and OfflineLogDirs=false > > The broker is then reconfigured to remove t-2 from log.dirs, and at startup, > the registration request shows OnlineLogDirs=[d1] and OfflineLogDirs=false. > The previous registration will only be replaced after a new successful > registration, regardless of how quickly or how often b1 restarts. > The controller compares the previous registration, and notices > that one of the directories has been removed. > So for any replica hosted in the broker that is assigned to that > missing log directory, a logical metadata update takes place > that assigned them to Uuid.OfflineDir, so Assignment.Directory > is updated for t-2. This value is indicates that the replica > is offline — I have updated the section you quoted to address this. > > Once the broker catches up with metadata, it will select the only > configured log directory — d1 — for any partitions assigned to > Uuid.OfflineDir, and update the assignment. > > Best, > > -- > Igor > > >
