I think JBOD definitely needs to be before 4.0. That has been a blocker issue this entire time for me and my team and I'm sure others. While Kraft has been technically "production ready" for a while, I haven't been able to upgrade because of missing JBOD support.
On Wed, Oct 11, 2023 at 12:15 PM Ismael Juma <m...@ismaeljuma.com> wrote: > Hi Luke, > > This is a good discussion. And there is a lot more to it than KRaft. > > With regards to KRaft, there are two separate items: > 1. Bugs > 2. Missing features when compared to ZK > > When it comes to bugs, I don't see why 4.0 is particularly relevant. KRaft > has been considered production-ready for over a year. If the bug is truly > critical, we should fix it for 3.6.1 or 3.7.0 (depending on the > complexity). > > When it comes to missing features, it would be preferable to land them > before 4.0 as well (ideally 3.7). I believe KIP-858 (JBOD) is the obvious > one in this category, but there are a few more in your list worth > discussing. > > Ismael > > On Wed, Oct 11, 2023 at 5:18 AM Luke Chen <show...@gmail.com> wrote: > > > Hi all, > > > > While Kafka 3.6.0 is released, I’d like to start the discussion for the > > “road to Kafka 4.0”. Based on the plan in KIP-833 > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.7 > > >, > > the next release 3.7 will be the final release before moving to Kafka 4.0 > > to remove the Zookeeper from Kafka. Before making this major change, I'd > > like to get consensus on the "must-have features/fixes for Kafka 4.0", to > > avoid some users being surprised when upgrading to Kafka 4.0. The intent > is > > to have a clear communication about what to expect in the following > months. > > In particular we should be signaling what features and configurations are > > not supported, or at risk (if no one is able to add support or fix known > > bugs). > > > > Here is the JIRA tickets list > > <https://issues.apache.org/jira/issues/?jql=labels%20%3D%204.0-blocker> > I > > labeled for "4.0-blocker". The criteria I labeled as “4.0-blocker” are: > > 1. The feature is supported in Zookeeper Mode, but not supported in KRaft > > mode, yet (ex: KIP-858: JBOD in KRaft) > > 2. Critical bugs in KRaft, (ex: KAFKA-15489 : split brain in KRaft > > controller quorum) > > > > If you disagree with my current list, welcome to have discussion in the > > specific JIRA ticket. Or, if you think there are some tickets I missed, > > welcome to start a discussion in the JIRA ticket and ping me or other > > people. After we get the consensus, we can label/unlabel it afterwards. > > Again, the goal is to have an open communication with the community about > > what will be coming in 4.0. > > > > Below is the high level category of the list content: > > > > 1. Recovery from disk failure > > KIP-856 > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-856:+KRaft+Disk+Failure+Recovery > > >: > > KRaft Disk Failure Recovery > > > > 2. Prevote to support controllers more than 3 > > KIP-650 > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-650%3A+Enhance+Kafkaesque+Raft+semantics > > >: > > Enhance Kafkaesque Raft semantics > > > > 3. JBOD support > > KIP-858 > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft > > >: > > Handle > > JBOD broker disk failure in KRaft > > > > 4. Scale up/down Controllers > > KIP-853 > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-853%3A+KRaft+Controller+Membership+Changes > > >: > > KRaft Controller Membership Changes > > > > 5. Modifying dynamic configurations on the KRaft controller > > > > 6. Critical bugs in KRaft > > > > Does this make sense? > > Any feedback is welcomed. > > > > Thank you. > > Luke > > >
