Re: KRaft Migration and Kafka Controller behaviour

2024-03-20 Thread Paolo Patierno
Hi Sanaa, I actually ran a migration twice. First locally just following the procedure described by the official Kafka documentation https://kafka.apache.org/documentation/#kraft_zk_migration and then on Kubernetes, because I notice you are talking about StatefulSet. But in this case I used the Str

Re: KRaft Migration and Kafka Controller behaviour

2024-03-19 Thread Sanaa Syed
Hi Paolo, Thank you for your response! I tested out a different theory today where I deployed the kraft controller statefulset and waited to see which brokers would be elected as controllers. Here is an example of my migration right after I have provisioned the kraft controller brokers/statefulse

Re: KRaft Migration and Kafka Controller behaviour

2024-03-19 Thread Paolo Patierno
Hi Sanaa, from my experiece about running migration it never happened to me and it should not happen anyway. When a (ZooKeeper-based) broker registers to be the controller at the beginning, you can see that the corresponding /controller znode will have an -1 as epoch. Something like: {"version":2,

KRaft Migration and Kafka Controller behaviour

2024-03-18 Thread Sanaa Syed
Hello, I've begun migrating some of my Zookeeper Kafka clusters to KRaft. A behaviour I've noticed twice across two different kafka cluster environments is after provisioning a kraft controller quorum in migration mode, it is possible for a kafka broker to become an active controller alongside a k