You are right. If all replicas happen to fail at the same time, then even with ack=everyone your acked sent messages may still be lost. As Kafka replication documents stated, with N replicas it can tolerate N-1 concurrent failures, hence if you are really unlucky and get N concurrent failures then replication will not be prevent data loss.
Guozhang On Tue, Feb 28, 2017 at 5:54 PM, Jiecxy <253441...@qq.com> wrote: > Hi Guozhang, > > Thanks for your reply. I’m still confused. I checked the source code.Kafka > just uses the class FileChannel.write(buffer) to write the data on the > broker, it puts the data in the memory instead of disk. Only you set the > FileChannel.force(true), it will flush the data to the disk. I understand > the ack mode, but it just requires the isr to receive the data then return > the response, in fact, it doesn’t require them to flush the data to the > disk. Am I right? Anything wrong? > So my question is, since the flush appears only after called explicitly > (or relying on the os), so it is still possible for kafka to loss data > (like broker down before the data are flushed to the disk)? > > Thanks > Chen > > > 在 2017年2月27日，02:52，Guozhang Wang <wangg...@gmail.com> 写道： > > > > Hello Chen, > > > > Kafka flushes data to disk (i.e. fsync) asynchronously. Based on the > > ack.mode it will return the response of the produce request to producer > > after it has been replicated (likely in memory) on N partition replicas. > > > > > > Guozhang > > > > On Sun, Feb 26, 2017 at 1:39 AM, Jiecxy <253441...@qq.com> wrote: > > > >> Hi guys, > >> > >> Does kafka send the acks response to the producer after flush the > >> messages to the disk or just keep them in the memory? > >> How does Kafka flush the messages? By calling the system call, like > >> fsync()? > >> > >> Thanks > >> Chen > >> > >> > > > > > > -- > > -- Guozhang > > > > -- -- Guozhang
