On Sun, Oct 11, 2015 at 2:34 PM, Todd Palino <tpal...@gmail.com> wrote:
> To answer the question, yes, it is incorrect. There are a few things you > can do to minimize problems. One is to disable unclean leader election, use > acks=-1 on the producers, have an RF of 3 or greater, and set the min ISR > to 2. This means that the topic will only be available if there are at > least 2 replicas in sync, your producers will all wait for acknowledgements > from all in sync replicas (therefore, at least 2) before considering > produce requests to be complete, and if you get in a situation where all > three replicas go down, the cluster will not perform an unclean leader > election (which can lose messages). > > Basically, you have to trade availability for correctness here. You get to > pick one. > Thanks. I figured as much, but its good to have official confirmation. It may be good to clarify the section of the documentation I quoted least folks get the wrong impression, as the reality is that regardless of how high the replication factor is Kafka can lose messages with a single node failure if the in sync replica set is allowed to shrink to a single member. Cheers.
