Hi,
I have a Linux box running a software RAID5 array of 3 IDE disks. Each
disk, by itself, has a raw read bandwidth of over 40MB/sec. I thus
expect the raw read bandwidth of the RAID array to be at least 80MB/sec;
but it's just 26MB/sec. Any idea what could be wrong?
Details follow.
/proc/mdstat:
Personalities : [raid1] [raid5]
md1 : active raid5 hda6[0] hdc6[1] hdg6[2]
153806080 blocks level 5, 128k chunk, algorithm 2 [3/3] [UUU]
Raw bandwidth was measured using both "zcav /dev/..." and dd+iostat.
Smaller chunk sizes don't make a significant difference.
The >40MB/sec per disk also holds when simultaneously reading from all
disks directly (e.g., via three instances of zcav).
Each drive is connected as master on a separate IDE channel with no
slave (I use a PIC0680-based PCI IDE controller for the extra channel).
Using only the on-motherboard IDE with two drives sharing an IDE channel
provides similar results.
The 3 drives are of different brands, all 7200RPM, all ATA/100 or better.
Lookahead is set via "hdparm -a128 -A1" on the /dev/hd? devices and also
on /dev/md1. All values from -a16 and larger give similar results.
CPU usage is low, nothing else is accessing the disks, and there's no
array sync in progress.
Running Linux 2.6.8.1 on Fedora Core 2.
Curiously, if I remove one of the drives from the array
("mdadm --fail ...") and run in degraded mode, the bandwidth greatly
increases (to nearly the expected 80MB/sec), regardless of which drive
is removed. Likewise, RAID4 is significantly faster (but presumably has
worse read performance under multi-process load).
One theory is that the reads are slowed in non-degraded RAID5 because
they're not perfectly continuous (the parity chunks are skipped). But I
think a huge lookahead on the raw devices (e.g., "hdparm -a512
/dev/hd?") would have fixed that. It doesn't.
Any ideas?
Eran
=================================================================
To unsubscribe, send mail to [EMAIL PROTECTED] with
the word "unsubscribe" in the message body, e.g., run the command
echo unsubscribe | mail [EMAIL PROTECTED]
