Thanks for putting this together, David. A few questions (maybe some of them overlap with Jun's?)
1. Rather than having MigrationCheckRequest, can we just add a new JSON field to the broker registration like "kraftReady": true? After all, we are already going to have to read the broker registration from ZK as we discussed earlier (and as the KIP mentions). 2. Do we really need kafka.metadata.migration.enable? It's not clear to me what harm would result if we just left it out of the design. Presumably standing up a new KRaft controller quorum and pointing zk.connect at the old ZK cluster is a very intentional action without much chance that we did it accidentally. 3. We should probably be explicit that the new metadata.version the KIP mentions corresponds to an inter.broker.protocol of 3.4 or later (since we're targetting 3.4) and that using IBP 3.4 means ALWAYS forwarding. I realize that's implicit if you read some of the earlier KIPs, but it would be good to make it clear. Also presumably the controller will copy its statically configured value of inter.broker.protocol into the metadata.version used in the new log, so it all matches up. Again, probably clear to us but good to spell out. 4. Migration state names: should "ZooKeeper" be renamed to "MigrationIneligible"? Since it does mean you can't upgrade (in effect) 5. If we get a new ineligible broker registering in ZK after the migration begins, I assume we will just have to ignore it and keep going. This would be an operator error, of course, and this broker will not be able to participate in the cluster. It would be good to spell out that behavior a bit. best, Colin On Tue, Oct 4, 2022, at 15:19, David Arthur wrote: > Hey folks, I wanted to get the ball rolling on the discussion for the > ZooKeeper migration KIP. This KIP details how we plan to do an online > migration of metadata from ZooKeeper to KRaft as well as a rolling > upgrade of brokers to KRaft mode. > > The general idea is to keep KRaft and ZooKeeper in sync during the > migration, so both types of brokers can exist simultaneously. Then, > once everything is migrated and updated, we can turn off ZooKeeper > writes. > > This is a pretty complex KIP, so please take a look :) > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-866+ZooKeeper+to+KRaft+Migration > > Thanks! > David
