Stan, Quite a few people have been asking the same question. I created this wiki ( https://cwiki.apache.org/confluence/display/KAFKA/Changes+in+Kafka+0.8) to document the significant changes in 0.8.
The replication design in 0.8 leveraged some ideas in the paper that you mentioned, but is not the same. The key difference is that in 0.8, we tolerate f-1 failures with f replicas. In contrast, typical Paxos-based algorithms only tolerate f failures with 2*f + 1 replicas. In terms of release schedule, we have seen some blocker issues when testing 0.8 on some production data at LinkedIn. Those blockers are being tracked in https://cwiki.apache.org/confluence/display/KAFKA/Kafka+replication+development. We plan to release a beta version once those blockers are resolved, hopefully in the next few weeks. Thanks, Jun On Sat, Jan 19, 2013 at 4:38 PM, Stan Rosenberg <stan.rosenb...@gmail.com>wrote: > Hi, > > Could someone please comment on the state of replication in 0.8, i.e., > what's implemented and what remains to be implemented? On a first > blush I see similarity between what's in the > high-level documentation and Jun's paper, "Using Paxos to Build a > Scalable, Consistent, and Highly Available Datastore". I'd be > especially grateful if someone could highlight the differences. > > Thanks! > > stan >
