By replica == 1 do you mean replication-factor == 1 or something different?
You should have replication-factor == 3 if you are trying to have durable writes survive failure. On the producer side set ack = -1 with that for it to work as expected. On Wed, Dec 10, 2014 at 7:14 PM, Helin Xiang <xkee...@gmail.com> wrote: > Thanks for the reply , Joe. > > In my opinion, when replica == 1, the ack == -1 would cause producer > stopping sending any data to kafka cluster if 1 broker is down. That means > we could not bear single point of failure. Am I right? > > What we want is when 1 broker down, and the topic replica is set to 1, the > whole system is still available and the data would go to other partitions > without loss. > > > THANKS again. > > On Thu, Dec 11, 2014 at 12:37 AM, Joe Stein <joe.st...@stealth.ly> wrote: > > > If you want no data loss then you need to set ack = -1 > > Copied from https://kafka.apache.org/documentation.html#producerconfigs > == > > -1, which means that the producer gets an acknowledgement after all > in-sync > > replicas have received the data. This option provides the best > durability, > > we guarantee that no messages will be lost as long as at least one in > sync > > replica remains. > > > > /******************************************* > > Joe Stein > > Founder, Principal Consultant > > Big Data Open Source Security LLC > > http://www.stealth.ly > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > ********************************************/ > > > > On Wed, Dec 10, 2014 at 11:27 AM, Helin Xiang <xkee...@gmail.com> wrote: > > > > > Hi, > > > > > > in some topics of our system, the data volumn is so huge that we think > > > doing extra replica is a waste of disk and network resource( plus the > > data > > > is not so important). > > > > > > firstly, we use 1 replica + ack=0, found when 1 broker is down, the > data > > > would loss 1/n. > > > then we tried 1 replica + ack=1, and found after 3 tries, the data is > > still > > > lost. and when we set the try number large enough, no more data can be > > > produced to Kafka. > > > > > > If I did not misunderstand, In 0.7, when 1 broker is down, both > producing > > > and consuming are available with no data loss. I can see the reason why > > > kafka 0.8 is designed to be different like 0.7. but is there a way to > > let > > > producer of 0.8 act like the behavior in 0.7? We don't care which part > > of > > > data should go to the specific partition, as long as the data goes into > > the > > > kafka with no loss. > > > > > > > > > Thanks > > > > > > > > > > > > > > > -- > > > > > > > > > *Best RegardsXiang Helin* > > > > > > > > > -- > > > *Best Regards向河林* >
