On 7/22/2012 6:41 AM, Shaffin Bhanji wrote: > Hello, > > I am trying to put together a 2U storage server for data. I have > previously invested in NAS equipment such as the Netgear NAS 1100 that I > have been disappointed in to say the least - data write speed of 5MB/s. > > This time around I want to build something that I have control over > hardware than to rely on equipment that I am locked in with, and not to > mention limitations. > > I would like an opinion from this group on successful implementation as > I will highly be using the server for vitalization disks (iSCSI), > backup, file share, etc. I want to make sure that I chose the right > hardware to get the best read/write performance. > > Experiences shared will greatly be appreciated.
If building a NFS/CIFS server from channel parts, I recommend the following. Newegg only ships to N. America but these parts are top shelf and common, and most if not all should be available from a local vendor worldwide. http://www.newegg.com/Product/Product.aspx?Item=N82E16811152478 http://www.newegg.com/Product/Product.aspx?Item=N82E16813182245 http://www.newegg.com/Product/Product.aspx?Item=N82E16819105277 http://www.newegg.com/Product/Product.aspx?Item=N82E16835114078 4x http://www.newegg.com/Product/Product.aspx?Item=N82E16820148382 http://www.newegg.com/Product/Product.aspx?Item=N82E16816118169 http://www.newegg.com/Product/Product.aspx?Item=N82E16816118162 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16816116098 8x http://www.newegg.com/Product/Product.aspx?Item=N82E16822236243 http://www.newegg.com/Product/Product.aspx?Item=N82E16827151242 Summary: 4 core Opteron @2.2GHz, 16GB ECC DDR1333, dual GbE LSI dual core RAID PCIe x8 1GB DDR1333 BBWC 8x 10k RPM 1TB WD Raptor drives in hot swap This hardware gives you the most performance bang for the buck in a 2U chassis. Configure the drives as a single RAID10 array in the LSI webBIOS, yielding 4TB of usable space. Use a relatively small strip (not stripe) size of 64KB, for a stripe size of 4x64= 256KB. This will yield the best read performance for virtual machine disk files and general file sharing. With the battery module you will enable write back caching, which will dramatically increase write performance for all workloads due to the 1GB of cache. Create a single 4TB virtual disk of the array and export it a single LUN. Configure the mobo BIOS to boot from the LSI controller. Boot your install DVD, create a small partition for /boot (10MB), one for the root filesystem (50GB), and install the bootloader (I prefer LILO for hardware RAID) in the MBR. Create a 4GB partition as swap. Pick whatever filesystem you prefer for /boot and the root filesystem. I normally use EXT2 for /boot, and EXT3/4 for the root filesystem. Finish your Debian install. Once finished, and assuming the reboot from the controller LUN works fine, create a partition of the remaining space on the LUN. Then create one large XFS filesystem of the remaining ~3.9TB of space, with this command: ~$ mkfs.xfs -d su=64k,sw=4 /dev/sda4 The su and sw parameters are somewhat critical, as they align all writes to the hardware stripe width of the RAID device, allowing the hardware write cache to perform optimally. Edit fstab and mount the XFS filesystem as such: /dev/sda4 /data xfs nobarrier,inode64 0 0 Since you have BBWC, nobarrier tells XFS to not use write barriers when writing to the journal or when doing unwritten extent conversion. This can speed up metadata (directory) operations significantly, as XFS will not flush the hardware write cache, as it would normally do without this setting. inode64 changes the allocator behavior, putting metadata and files in the same allocation group, decreasing read/write latency. The default inode32 allocator puts all metadata in AG0, causing head thrashing and increases latency with common workloads. Now create your data directories, virtual machine disk files, and iSCSI LUN target exports as you normally would, along with your samba shares. And enjoy the phenomenal IO performance this relatively inexpensive 8 drive machine will give you. If 4TB isn't enough, you can use 2TB RE4 drives: http://www.newegg.com/Product/Product.aspx?Item=N82E16822136579 and double your storage to 8T. Due to the 1GB of BBWC on the LSI your write performance may not decrease much, but your read performance will drop by about 50%. The RE4s will also save about $400 on the 8 drives, at Newegg pricing, but given the overall cost of the machine that's not much savings. If you saying to yourself "I'll just use RAID6 and get 2 more TB from the fast drives". If you do this I've just wasted more than an hour of my time here. Using RAID6 will cause a read-modify-write cyle every time you modify an existing file--which is all the time with virtual machine disk files. That RMW cycle increases latency per write operation by anywhere from 6-10 fold and decreases write throughput by about the same. The latest generation of LSI controllers, including the 9266-8i, do have pretty fantastic performance with RAID6 RMW due to the large cache and fast ASICs, but the write performance under heavy load is still less than half that of RAID10, which isn't affected by RMW--there's no parity to recalculate. Hope you've found this information and useful, given the time I spent putting it all together for you. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/500d14f5.4020...@hardwarefreak.com
