--- On Mon, 7/13/09, Maxim Khitrov <mkhit...@gmail.com> wrote: > From: Maxim Khitrov <mkhit...@gmail.com> > Subject: Re: ZFS or UFS for 4TB hardware RAID6? > To: mahle...@yahoo.com > Cc: "Free BSD Questions list" <freebsd-questions@freebsd.org> > Date: Monday, July 13, 2009, 3:23 PM > On Mon, Jul 13, 2009 at 2:13 PM, > Richard Mahlerwein<mahle...@yahoo.com> > wrote: > > > > --- On Mon, 7/13/09, Maxim Khitrov <mkhit...@gmail.com> > wrote: > > > >> From: Maxim Khitrov <mkhit...@gmail.com> > >> Subject: Re: ZFS or UFS for 4TB hardware RAID6? > >> To: mahle...@yahoo.com > >> Cc: "Free BSD Questions list" <freebsd-questions@freebsd.org> > >> Date: Monday, July 13, 2009, 2:02 PM > >> On Mon, Jul 13, 2009 at 1:46 PM, > >> Richard Mahlerwein<mahle...@yahoo.com> > >> wrote: > >> >> > >> >> Your mileage may vary, but... > >> >> > >> >> I would investigate either using more > spindles if > >> you want > >> >> to stick to RAID6, or perhaps using > another RAID > >> level if > >> >> you will be with 4 drives for a while. > The > >> reasoning > >> >> is that there's an overhead with RAID 6 - > parity > >> blocks are > >> >> written to 2 disks, so in a 4 drive > combination > >> you have 2 > >> >> drives with data and 2 with parity. > >> >> > >> >> With 4 drives, you could get much, much > higher > >> performance > >> >> out of RAID10 (which is alternatively > called > >> RAID0+1 or > >> >> RAID1+0 depending on the manufacturer and > on how > >> accurate > >> >> they wish to be, and on how they > actually > >> implemented it, > >> >> too). This would also mean 2 usable > drives, as > >> well, so > >> >> you'd have the same space available in > RAID10 as > >> your > >> >> proposed RAID6. > >> >> > >> >> I would confirm you can, on the fly, > convert from > >> RAID10 to > >> >> RAID6 after you add more drives. If you > can not, > >> then > >> >> by all means stick with RAID6 now! > >> >> > >> >> With 4 1 TB drives (for simpler > examples) > >> >> RAID5 = 3 TB available, 1 TB worth used > in > >> "parity". > >> >> Fast reads, slow writes. > >> >> RAID6 = 2 TB available, 2 TB worth used > in > >> "parity". > >> >> Moderately fast reads, slow writes. > >> >> RAID10 = 2 TB available, 2TB in duplicate > copies > >> (easier > >> >> work than parity calculations). Very > fast > >> reads, > >> >> moderately fast writes. > >> >> > >> >> When you switch to, say, 8 drives, the > numbers > >> start to > >> >> change a bit. > >> >> RAID5 = 7TB available, 1 lost. > >> >> RAID6 = 6TB available, 2 lost. > >> >> RAID10 = 4TB available, 4 lost. > >> >> > >> > > >> > Sorry, consider myself chastised for having > missed the > >> "Security is more important than performance" bit. > I tend > >> toward solutions that show the most value, and > with 4 > >> drives, it seems that I'd stick with the same > "data > >> security" only pick up the free speed of RAID10. > Change > >> when you get to 6 or more drives, if necessary. > >> > > >> > For data security, I can't answer for the > UFS2 vs. > >> ZFS. For hardware setup, let me amend everything > I said > >> above with the following: > >> > > >> > Since you are seriously focusing on data > integrity, > >> ignore everything I said but make sure you have > good > >> backups! :) > >> > > >> > Sorry, > >> > -Rich > >> > >> No problem :) I've been doing some reading since I > posted > >> this > >> question and it turns out that the controller will > actually > >> not allow > >> me to create a RAID6 array using only 4 drives. > 3ware > >> followed the > >> same reasoning as you; with 4 drives use RAID10. > >> > >> I know that you can migrate from one to the other > when a > >> 5th disk is > >> added, but RAID10 can only handle 2 failed drives > if they > >> are from > >> separate RAID1 groups. In this way, it is just > slightly > >> less resilient > >> to failure than RAID6. With this new information, > I think I > >> may as > >> well get one more 2TB drive and start with 6TB of > RAID6 > >> space. This > >> will be less of a headache later on. > >> > >> - Max > > > > Just as a question: how ARE you planning on backing > this beast up? While I don't want to sound like a > worry-wort, I have had odd things happen at the worst of > times. RAID cards fail, power supplies let out the magic > smoke, users delete items they really want back... *sigh* > > Rsync over ssh to another server. Most of the data stored > will never > change after the first upload. A daily rsync run will > transfer one or > two gigs at the most. History is not required for the same > reason; > this is an append-only storage for the most part. A backup > for the > previous day is all that is required, but I will keep a > weekly backup > as well until I start running out of space. > > > A bit of reading shows that ZFS, if it's stable > enough, has some really great features that would be nice on > such a large pile o' drives. > > > > See http://wiki.freebsd.org/ZFSQuickStartGuide > > > > I guess the last question I'll ask (as any more may > uncover my ignorance) is if you need to use hardware RAID at > all? It seems both UFS2 and ZFS can do software RAID > which seems to be quite reasonable with respect to > performance and in many ways seems to be more robust since > it is a bit more portable (no specialized hardware). > > I've thought about this one a lot. In my case, the hard > drives are in > a separate enclosure from the server and the two had to be > connected > via SAS cables. The 9690SA-8E card was the best choice I > could find > for accessing an external SAS enclosure with support for 8 > drives. > > I could configure it in JBOD mode and then use software to > create a > RAID array. In fact, I will likely do this to compare > performance of a > hardware vs. software RAID5 solution. The ZFS RAID-Z option > does not > appeal to me, because the read performance does not benefit > from > additional drives, and I don't think RAID6 is available in > software. > For those reasons I'm leaning toward a hardware > implementation. > > If I go the hardware route, I'll try to purchase a backup > controller > in a year or two. :) > > > There are others who may respond with better > information on that front. I've been a strong > proponent of hardware RAID, but have recently begun to > realize many of the reasons for that are only of limited > validity now. > > Agreed, and many simple RAID setups (0, 1, 10) will give > you much > better performance in software. In my case, I have to have > some piece > of hardware just to get to the drives, and I'm guessing > that hardware > RAID5/6 will be faster than the closest software > equivalent. Maybe my > tests will convince me otherwise. > > - Max
I'd love to hear about any test results you may get comparing software with hardware raid. _______________________________________________ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
