Hi Eric - thanks for your reply.
Yes, zpool iostat -v
I've re-configured the setup into two pools for a test:
1st pool: 8 disk stripe vdev
2nd pool: 8 disk stripe vdev
The SSDs are currently not in the pool since I am not even reaching what the
spinning rust is capable of - I believe I have a deeper issue and they would
only complicate things for me at this point.
I can reconfigure the pool however needed, since this server is not yet in
production.
My test is through 8gb FC target through comstar from a Windows Workstation.
The pool is currently configured with a default 128k recordsize.
Then I:
touch /pool/file
stmfadm create-lu -p wcd=false -s 10T /pool/file
stmfadm add-view <lu>
(The lu defaults to reporting a 512 blk size)
I formatted the volume NTFS cluster size default 4k
I do that twice (two seperate pools, two seperate LUNs, etc)
Then I copy a large file (700MB or so) to one of the LUNs from the local
workstation.
The read performance of my workstation harddrive is about 100+ MBps, and as
such the file copies at about that speed.
Then I make a few copies of the file on that LUN so that I have about 20+ GB of
that same file on one of the LUNs.
Then I reboot the opensolaris server (since the cache is nicely populated at
this point and everything is running fast)
Then I try copying the lot of those files from one lun to the other.
The read performance appears to be limiting my write performance.
I have tried matching recordsize to NTFS cluster size at 4k, 16k, 32 and 64k.
I have tried making NTFS clustersize a multiple of recordsize.
I have seen performance improvements as a result (I dont' have numbers)
however, none of the cluster/block combinations brought me to where I should be
on reads.
I've tried many configurations - and I've seen my performance fluctuate up and
down here and there. However, it's never on-par with what it should be and the
reads seem to be a limiting factor.
For clarity - here's some 'zpool iostat -v 1' output from my current
configuration directly following a reboot of the server
while copying 13GB of those files from LUN -> LUN:
capacity operations bandwidth
pool alloc free read write read write
----------------------- ----- ----- ----- ----- ----- -----
~snip~
edit1 13.8G 16.3T 773 0 96.5M 0
c0t5000C50020C7A44Bd0 1.54G 1.81T 75 0 9.38M 0
c0t5000C50020C7C9DFd0 1.54G 1.81T 89 0 11.2M 0
c0t5000C50020C7CE1Fd0 1.53G 1.81T 82 0 10.3M 0
c0t5000C50020C7D86Bd0 1.53G 1.81T 85 0 10.6M 0
c0t5000C50020C61ACBd0 1.55G 1.81T 83 0 10.4M 0
c0t5000C50020C79DEFd0 1.54G 1.81T 92 0 11.5M 0
c0t5000C50020CD3473d0 1.53G 1.81T 84 0 10.6M 0
c0t5000C50020CD5873d0 1.53G 1.81T 87 0 11.0M 0
c0t5000C500103F36BFd0 1.54G 1.81T 92 0 11.5M 0
----------------------- ----- ----- ----- ----- ----- -----
syspool 35.1G 1.78T 0 0 0 0
mirror 35.1G 1.78T 0 0 0 0
c0t5000C5001043D3BFd0s0 - - 0 0 0 0
c0t5000C500104473EFd0s0 - - 0 0 0 0
----------------------- ----- ----- ----- ----- ----- -----
test1 11.0G 16.3T 850 0 106M 0
c0t5000C500103F48FFd0 1.23G 1.81T 95 0 12.0M 0
c0t5000C500103F49ABd0 1.23G 1.81T 92 0 11.6M 0
c0t5000C500104A3CD7d0 1.22G 1.81T 92 0 11.6M 0
c0t5000C500104A5867d0 1.24G 1.81T 97 0 12.0M 0
c0t5000C500104A7723d0 1.22G 1.81T 95 0 11.9M 0
c0t5000C5001043A86Bd0 1.23G 1.81T 96 0 12.1M 0
c0t5000C5001043C1BFd0 1.22G 1.81T 91 0 11.3M 0
c0t5000C5001043D1A3d0 1.23G 1.81T 91 0 11.4M 0
c0t5000C5001046534Fd0 1.23G 1.81T 97 0 12.2M 0
----------------------- ----- ----- ----- ----- ----- -----
~snip~
Here's some zpool iostat (no -v) output over the same time:
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
~snip~
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 0 956 0 120M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 142 564 17.9M 52.8M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 723 0 90.3M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 1.37K 0 174M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 536 0 66.9M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 980 0 122M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 958 0 120M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 771 0 96.0M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 899 0 112M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 1.00K 0 128M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 968 0 121M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 256 0 32.0M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 305 690 38.2M 81.6M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 255 0 32.0M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 151 868 19.0M 109M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 47 933 5.99M 117M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 0 953 0 119M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 0 955 0 119M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 256 0 32.0M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 281 443 35.2M 46.4M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 537 0 67.0M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 780 0 97.4M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 458 0 56.9M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 837 0 105M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 1.67K 0 212M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 481 0 60.1M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 773 0 96.6M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 769 0 96.0M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 682 0 85.2M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 779 0 97.2M 0
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 237 0 29.7M 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 493 518 61.6M 56.6M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 55 788 6.99M 98.4M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 50 927 6.37M 116M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 0 969 0 121M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 0 965 0 121M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 25 707 3.25M 72.6M
---------- ----- ----- ----- ----- ----- -----
edit1 13.8G 16.3T 0 0 0 0
syspool 35.1G 1.78T 0 0 0 0
test1 11.9G 16.3T 108 11 13.6M 12.0K
~snip~
--
This message posted from opensolaris.org
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
