On Fri, 15 Feb 2008, Luke Lonergan wrote:
I'm assuming you're measuring sequential write speed posting the iozone
results would help guide the discussion.
Posted below. I am also including the output from mpathadm in case
there is something wrong with the load sharing.
For the configuration you describe, you should definitely be able to sustain
200 MB/s write speed for a single file, single thread due to your use of
4Gbps Fibre Channel interfaces and RAID1. Someone else brought up that with
I only managed to get 200 MB/s write when I did RAID 0 across all
drives using the 2540's RAID controller and with ZFS on top.
host based mirroring over that interface you will be sending the data twice
over the FC-AL link, so since you only have 400 MB/s on the FC-AL interface
(load balancing will only work for two writes), then you have to divide that
by two.
While I agree that data is sent twice (actually up to 8X if striping
across four mirrors), it seems to me that the load balancing should
still work for one application write since ZFS is what does the
multiple device I/Os.
If you do the mirroring on the RAID hardware you¹ll get double that speed on
writing, or 400MB/s and the bottleneck is still the single FC-AL interface.
I didn't see that level of performance. Perhaps there is something I
should be investigating?
Bob
Output of 'mpathadm list lu':
/scsi_vhci/[EMAIL PROTECTED]
Total Path Count: 1
Operational Path Count: 1
/scsi_vhci/[EMAIL PROTECTED]
Total Path Count: 1
Operational Path Count: 1
/dev/rdsk/c4t600A0B80003A8A0B0000096147B451BEd0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B800039C9B500000A9C47B4522Dd0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B800039C9B500000AA047B4529Bd0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B80003A8A0B0000096647B453CEd0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B800039C9B500000AA447B4544Fd0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B80003A8A0B0000096A47B4559Ed0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B800039C9B500000AA847B45605d0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B80003A8A0B0000096E47B456DAd0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B800039C9B500000AAC47B45739d0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B800039C9B500000AB047B457ADd0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B80003A8A0B0000097347B457D4d0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c4t600A0B800039C9B500000AB447B4595Fd0s2
Total Path Count: 2
Operational Path Count: 2
Output of 'mpathadm show lu /dev/rdsk/c4t600A0B800039C9B500000AB047B457ADd0s2':
Logical Unit: /dev/rdsk/c4t600A0B800039C9B500000AB047B457ADd0s2
mpath-support: libmpscsi_vhci.so
Vendor: SUN
Product: LCSM100_F
Revision: 0617
Name Type: unknown type
Name: 600a0b800039c9b500000ab047b457ad
Asymmetric: yes
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA
Paths:
Initiator Port Name: 10000000c967c830
Target Port Name: 200400a0b83a8a0c
Override Path: NA
Path State: OK
Disabled: no
Initiator Port Name: 10000000c967c82f
Target Port Name: 200500a0b83a8a0c
Override Path: NA
Path State: OK
Disabled: no
Target Port Groups:
ID: 4
Explicit Failover: yes
Access State: standby
Target Ports:
Name: 200400a0b83a8a0c
Relative ID: 0
ID: 1
Explicit Failover: yes
Access State: active
Target Ports:
Name: 200500a0b83a8a0c
Relative ID: 0
Performance test run using iozone:
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: Solaris10gcc-64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.
Run began: Thu Feb 14 16:35:51 2008
Auto Mode
Using Minimum Record Size 64 KB
Using Maximum Record Size 512 KB
Using minimum file size of 33554432 kilobytes.
Using maximum file size of 67108864 kilobytes.
Command line used: iozone -a -i 0 -i 1 -y 64 -q 512 -n 32G -g 64G
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random
bkwd record stride
KB reclen write rewrite read reread read write
read rewrite read fwrite frewrite fread freread
33554432 64 150370 113779 454731 456158
33554432 128 147032 181308 455496 456239
33554432 256 148182 169944 454192 456252
33554432 512 153843 194189 473982 516130
67108864 64 151047 111227 463406 456302
67108864 128 148597 159236 456959 488100
67108864 256 148995 165041 463519 453896
67108864 512 154556 166802 458304 456833
iozone test complete.
======================================
Bob Friesenhahn
[EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
