J.A. Magallón wrote: > On Wed, 18 Jul 2007 10:56:11 +0100, Rui Santos <[EMAIL PROTECTED]> wrote: > > >> Hi, >> >> I'm getting a strange slow performance behavior on a recently installed >> Server. Here are the details: >> >> > ... > >> I can get a write throughput of 60 MB/sec on each HD by issuing the >> command 'time `dd if=/dev/zero of=test.raw bs=4k count=$(( 1024 * 1024 / >> 4 )); sync`' >> >> > ... > >> The RAID device I'm testing on is /dev/md2. Now, by issuing the same >> command 'dd if=/dev/zero of=test.raw bs=4k count=$(( 1024 * 1024 / 4 )); >> sync`' on the raid device mount point, I get the following speeds: >> With stripe_cache_size at default '265': 51 MB/sec >> With stripe_cache_size at '8192': 73 MB/sec >> >> > > I know many people consider this stupid, but can you post some hdparm -tT > data ? >
Of course. Here's the output: NewServer-RD:~ # hdparm -tT /dev/md2 /dev/md2: Timing cached reads: 1738 MB in 2.00 seconds = 868.93 MB/sec Timing buffered disk reads: 444 MB in 3.01 seconds = 147.69 MB/sec NewServer-RD:~ # hdparm --direct -tT /dev/md2 /dev/md2: Timing O_DIRECT cached reads: 290 MB in 2.01 seconds = 144.05 MB/sec Timing O_DIRECT disk reads: 396 MB in 3.01 seconds = 131.75 MB/sec > The culprit can be the filesystem+pagecache, the md driver or the disk > driver, so I think trying just hdparm will show if the disk o md are > going nuts... > > In my case, I have a box with 2 raids, one with SCSI disks and one with > IDE ones. > > Some results: > > lsscsi: > [0:0:0:0] disk IBM DDYS-T18350N S96H /dev/sda > [2:0:0:0] disk SEAGATE ST336807LW 0C01 /dev/sdb > [2:0:1:0] disk SEAGATE ST336807LW 0C01 /dev/sdc > [2:0:2:0] disk SEAGATE ST336807LW 0C01 /dev/sdd > [2:0:3:0] disk SEAGATE ST336807LW 0C01 /dev/sde > [3:0:0:0] disk ATA ST3120022A 3.06 /dev/sdf > [3:0:1:0] cd/dvd HL-DT-ST DVDRAM GSA-4040B A300 /dev/sr0 > [4:0:0:0] disk ATA ST3120022A 3.76 /dev/sdg > > > /dev/md0: > Version : 00.90.03 > Creation Time : Mon Jun 18 13:40:57 2007 > Raid Level : raid5 > Array Size : 107522304 (102.54 GiB 110.10 GB) > Used Dev Size : 35840768 (34.18 GiB 36.70 GB) > Raid Devices : 4 > Total Devices : 4 > Preferred Minor : 0 > Persistence : Superblock is persistent > > Update Time : Wed Jul 18 13:31:22 2007 > State : clean > Active Devices : 4 > Working Devices : 4 > Failed Devices : 0 > Spare Devices : 0 > > Layout : left-symmetric > Chunk Size : 256K > > UUID : 51ad72a7:a4d20d15:0f3ea3a1:5ccb49a0 > Events : 0.2 > > Number Major Minor RaidDevice State > 0 8 17 0 active sync /dev/sdb1 > 1 8 33 1 active sync /dev/sdc1 > 2 8 49 2 active sync /dev/sdd1 > 3 8 65 3 active sync /dev/sde1 > > This is, four scsi disks on a Adaptec U320, doing raid5: > > /dev/sdb: > Timing cached reads: 904 MB in 2.00 seconds = 451.84 MB/sec > Timing buffered disk reads: 228 MB in 3.00 seconds = 75.90 MB/sec > /dev/sdc: > Timing buffered disk reads: 226 MB in 3.01 seconds = 75.01 MB/sec > /dev/sdd: > Timing buffered disk reads: 228 MB in 3.00 seconds = 75.88 MB/sec > /dev/sde: > Timing buffered disk reads: 226 MB in 3.00 seconds = 75.31 MB/sec > > /dev/md0: > Timing buffered disk reads: 562 MB in 3.01 seconds = 186.88 MB/sec > > Nearly 75x3 = 215 Mb/s. And this looks like a small regression, I remember > to have seen 200Mb on this setup on previous kernels. > Performance is like 186/215 = 86%. > > And /dev/md1, raid0 on 2 IDE disks: > > /dev/sdf: > Timing buffered disk reads: 148 MB in 3.02 seconds = 48.93 MB/sec > /dev/sdg: > Timing buffered disk reads: 124 MB in 3.00 seconds = 41.33 MB/sec > > /dev/md1: > Timing buffered disk reads: 204 MB in 3.01 seconds = 67.68 MB/sec > > Performance: 67 / 90 = 75%, more or less...not too good. > > Now that I read the hdparm man page, perhaps would be better to repeat > the tests with hdparm --direct. > > -- > J.A. Magallon <jamagallon()ono!com> \ Software is like sex: > \ It's better when it's free > Mandriva Linux release 2008.0 (Cooker) for i586 > Linux 2.6.21-jam12 (gcc 4.2.1 20070704 (4.2.1-3mdv2008.0)) SMP PREEMPT > 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0 > > > > Thanks for your reply. Rui Santos - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
