Hi Deng, Thanks for the question. I mentioned this in the rejected alternatives section. The current proposal is only for metadata, but I am definitely in favor of using Raft for partition replication in the long term as well. There are some interesting tradeoffs in terms of fault tolerance, latency, and batching compared with the current replication protocol. I consider this a good candidate for the next big architectural change once KIP-500 nears completion.
-Jason On Sun, Apr 19, 2020 at 7:16 PM deng ziming <dengziming1...@gmail.com> wrote: > Big +1 for your initiative and I have a question, we implement the > Raft protocol > just to be used in the management of metadata in Zookeeper or we will also > use it to replace the current logical of managing log-replica since the > algorithm we used to manage log-replica is analogous to Raft. > > > On Fri, Apr 17, 2020 at 7:45 AM Jason Gustafson <ja...@confluent.io> > wrote: > > > Hi All, > > > > I'd like to start a discussion on KIP-595: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum > > . > > This proposal specifies a Raft protocol to ultimately replace Zookeeper > as > > documented in KIP-500. Please take a look and share your thoughts. > > > > A few minor notes to set the stage a little bit: > > > > - This KIP does not specify the structure of the messages used to > represent > > metadata in Kafka, nor does it specify the internal API that will be used > > by the controller. Expect these to come in later proposals. Here we are > > primarily concerned with the replication protocol and basic operational > > mechanics. > > - We expect many details to change as we get closer to integration with > > the controller. Any changes we make will be made either as amendments to > > this KIP or, in the case of larger changes, as new proposals. > > - We have a prototype implementation which I will put online within the > > next week which may help in understanding some details. It has diverged a > > little bit from our proposal, so I am taking a little time to bring it in > > line. I'll post an update to this thread when it is available for review. > > > > Finally, I want to mention that this proposal was drafted by myself, > Boyang > > Chen, and Guozhang Wang. > > > > Thanks, > > Jason > > >
