On 2012-04-03, at 7:04 AM, Francois Chassaing wrote: > Hello list, > I'm trying to tune my lustreFS so that it'll perform better, because I'm sure > I have a lot of low-level stuff that needs to be optimzed. > My (poor) choice was to have a pair of MDS backed by a (supposely fast > Equallogic) iSCSI array, I also have three OSSes serving one OST each. > Those OST are a set of 6 NL SAS drives on RAID10 behind a Perc6/i (those Dell > cards I'm sure a lot know about, this one has BBU & 256Mb RAM). system for > those OST are on a separate mirrored pair of small disks > lnet sits on DDR IB (mellanox memory-less cards). Lustre version is 1.8.7-wc1 > > I want to use more sensible settings for the RAID volume : > First of all I would change the chunk/stripe/whatever-name-you-like size for > the volume to try to align with 1Mb/# of drives. > If I had four drives RAID10, i would have chosen 512k, since I've 6, I'm more > leaning towards 256 (means 1.33 writes per disk for 1Mb 'block' which would > most probably perform better than the 5.33 writes per disk using the 64k > default setting).
What you really should be looking at is to do component-level analysis using the Lustre iokit to test the performance of each part of your IO stack. First the RAID level, next the OST level, and finally the client performance. > Does that makes sense to any of you, or is this pointless ? > Also, given the Raid card, do you think that it will be better to really > align to 1Mb by using a 5-drive RAID5 (so sparing one drive) or a RAID6 > volume ? > then I could use the 256k stripe size AND be aligned, but I'll have to pay > the cost for parity-computation and write(s)... I suspect you will get better performance and reliability with RAID-6 4+2 than with RAID-10 3+3, but this is easily tested with sgpdd-survey from the iokit. Note also that most deployments have multiple OSTs per OSS, since 6 drives is typically not enough to saturate a decent server, and you get more performance with less overhead. You could probably put 4 OSTs into each OSS reasonably. At least it is something to consider for the future if you expand your filesystem. Cheers, Andreas -- Andreas Dilger Whamcloud, Inc. Principal Lustre Engineer http://www.whamcloud.com/ _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
