+1. I've tried this on Linux and it helps reduce the spikes in append (and hence producer) latency for high throughput writes. I am not entirely sure why but my suspicion is that in the absence of preallocation, you see spikes writes need to happen faster than the time it takes Linux to allocate the next block to the file.
It will be great to see some performance test results too. On Tue, Apr 21, 2015 at 9:23 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > I'm also +1 on this. The change is quite small and may actually help perf > on Linux as well (we've never tried this). > > I have a lot of concerns on testing the various failure conditions but I > think since it will be off by default the risk is not too high. > > -Jay > > On Mon, Apr 20, 2015 at 6:58 PM, Honghai Chen <honghai.c...@microsoft.com> > wrote: > > > I wrote a KIP for this after some discussion on KAFKA-1646. > > https://issues.apache.org/jira/browse/KAFKA-1646 > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-20+-+Enable+log+preallocate+to+improve+consume+performance+under+windows+and+some+old+Linux+file+system > > The RB is here: https://reviews.apache.org/r/33204/diff/ > > > > Thanks, Honghai > > > > > -- Thanks, Neha
