[
https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359658#comment-14359658
]
Honghai Chen commented on KAFKA-1646:
-------------------------------------
Hey [~jkreps] We have ask the NTFS owners, seemly this is the only way for NTFS
to optimize the performance. [~jghoman] also in the email thread. Below is
what they say in the email thread. Currently there are 7 "if
OS.IsWindows", can reduce to 3, will update the code review soon. And if
you agree that we can also pre allocate file with big size in Linux, we can
reduce the "if OS.IsWindows" to 0. Do you agree?
==============The answer from NTFS owners==========
""There’s no external way to influence NTFS’s allocation algorithms.
As suggested in the blog post, pre-allocating space in a file help a lot in
cases where an app does small extending writes to a file over a period of time
(a text-based log file being one canonical example), especially on a busy file
system where extending writes to other files are also happening.
All file systems require defragmenting. However Linux file systems (ext2/3/4
etc) implement the defragmenting inside the file system, while Windows file
systems implement the defragmenting using an external program. There are
trade-offs to both approaches. The external program (defrag.exe) is in a
scheduled task so it shouldn’t normally be necessary to ever run it by hand.""
===============================================================
http://www.howtogeek.com/115229/htg-explains-why-linux-doesnt-need-defragmenting/
https://stackoverflow.com/questions/29007319/how-to-make-windows-ntfs-does-not-have-fragment-or-reduce-fragmen
> Improve consumer read performance for Windows
> ---------------------------------------------
>
> Key: KAFKA-1646
> URL: https://issues.apache.org/jira/browse/KAFKA-1646
> Project: Kafka
> Issue Type: Improvement
> Components: log
> Affects Versions: 0.8.1.1
> Environment: Windows
> Reporter: xueqiang wang
> Assignee: xueqiang wang
> Labels: newbie, patch
> Attachments: Improve consumer read performance for Windows.patch,
> KAFKA-1646-truncate-off-trailing-zeros-on-broker-restart-if-bro.patch,
> KAFKA-1646_20141216_163008.patch, KAFKA-1646_20150306_005526.patch
>
>
> This patch is for Window platform only. In Windows platform, if there are
> more than one replicas writing to disk, the segment log files will not be
> consistent in disk and then consumer reading performance will be dropped down
> greatly. This fix allocates more disk spaces when rolling a new segment, and
> then it will improve the consumer reading performance in NTFS file system.
> This patch doesn't affect file allocation of other filesystems, for it only
> adds statements like 'if(Os.iswindow)' or adds methods used on Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
