Hello, There’s now a proposal to address ZK to KRaft migration — KIP-866 — but currently it doesn't address JBOD so I've decided to update this proposal to address that migration scenario.
So given that: - When migrating from a ZK cluster running JBOD to KRaft, brokers registering in KRaft mode will need to be able to register all configured log directories. - As part of the migration, the mapping of partition to log directory will have to be learnt by the active controller and persisted into the cluster metadata. - It isn’t safe to allow for leadership from replicas without this mapping, as if the hosting log directory fails there will be no failover mechanism. I have updated the proposal to reflect that: - Multiple log directories may be indicated in the first broker registration referencing log directory UUIDs. We no longer require a single log directory to start with. - The controller must never assign leadership to a replica in a broker registered with multiple log directories, unless the partition to log directory assignment is already in the cluster metadata. - The broker should not be unfenced until all of its partition to log directory mapping is persisted into cluster metadata I've also added details as to how the ZK to KRaft migration can work in a cluster that is already operating with JBOD. Please have a look and share your thoughts. -- Igor
