Hi Tao 1. I am wondering if the fsync operation is called by the last two routines internally? => Yes
2. If log.flush.interval.ms is not specified, is it true that Kafka let OS to handle pagecache flush in background? => Yes 3. If we specify ack=1 and ack=-1 in new producer, do those request only persist in pagecache or actual disk? => acknowledgements do not imply a channel is flushed. ack = -1 will increase durability through redundancy. If you really want to control fsync you can configure Kafka for force a flush after a defined number of messages using log.flush.interval.messages. I should add that this isn’t generally the best approach. You’ll get much better performance if you use multiple redundant replicas to manage your durability concerns, if you can. B > On 7 Aug 2015, at 05:49, Tao Feng <fengta...@gmail.com> wrote: > > Hi , > > I am trying to understand the Kafka log flush behavior. My understanding is > when the broker specifies broker config param "log.flush.interval.ms", it > will specify log config param "flush.ms" internally. In logManager logic, > when the log exceed flush.ms, it will call Log.flush which will call > FileChannel.force(true) and MappedByteBuffer.flush() . > > Couple of questions: > 1. I am wondering if the fsync operation is called by the last two routines > internally? > 2. If log.flush.interval.ms is not specified, is it true that Kafka let OS > to handle pagecache flush in background? > 3. If we specify ack=1 and ack=-1 in new producer, do those request only > persist in pagecache or actual disk? > > Thanks, > -Tao
