On 7/17/06, Jonathan Wheeler <[EMAIL PROTECTED]> wrote:
Hi All,
I've just built an 8 disk zfs storage box, and I'm in the testing phase before I put it into production. I've run into some unusual results, and I was hoping the community could offer some suggestions. I've bascially made the switch to Solaris on the promises of ZFS alone (yes I'm that excited about it!), so naturally I'm looking forward to some great performance - but it appears I'm going to need some help finding all of it.
One major concern Jonathan has is the 7-raidz write performance.
(I see no big surprise in 'read' results.)
"The really interesting numbers happen at 7 disks -
it's slower then with 4, in all tests."
I randomly picked 3 results from his several runs:
-Per Char- --Block--- -Rewrite--
MB K/sec %CPU K/sec %CPU K/sec %CPU
==== ========== ============ ==========
4-disk 8196 57965 67.9 123268 27.6 78712 17.1
7-disk 8196 49454 57.1 92149 20.1 73013 16.0
8-disk 8196 61345 70.7 139259 28.5 89545 20.8
I looked at the corresponding dtrace data for
7 and 8-raidz cases.
(Should have also asked for 4-raidz data.
Jonathan, you can still send 4-raidz data to me offline.)
In 7-raidz, each disk had writes in two sizes:
214-block or 85-block, equally.
DEVICE BLKs COUNT
-------- ---- --------
sd1 85 27855
214 27882
sd2 85 27854
214 27868
sd3 85 27849
214 27884
...
In 8-raidz,
sd1,3,5,7 had either 220 or 221-block writes, equally.
sd2,4,6,8 had 100% of 146-block writes.
DEVICE BLKs COUNT
-------- ---- --------
sd1 220 16325
221 16338
sd2 146 49001
sd3 220 16335
221 16333
sd4 146 49005
sd5 220 16340
221 16324
sd6 146 49001
sd7 220 16332
221 16333
sd8 146 49009
In terms of average write response time,
in 7-raidz
DEVICE WRITE AVG.ms
------- ------- ------
sd1 63990 54.03
sd2 64000 53.65
sd3 63898 55.48
sd4 64190 54.14
sd5 64091 54.81
sd6 63967 57.83
sd7 64092 54.19
in 8-raidz
DEVICE WRITE AVG.ms
------- ------- ------
sd1 42276 6.64
sd2 58467 19.66
sd3 42287 6.24
sd4 55198 20.01
sd5 42285 6.64
sd6 58409 22.90
sd7 42235 6.88
sd8 54967 24.46
At bdev level, 8-raidz shows much better turnaround
time than 7-raidz, while disk 1,3,5,7 (larger writes) are
better than 2,4,6,8 (smaller writes).
So 8-raidz wins by larger writes and much better response
time for each write, but why these two differences?
and why the disparity between odd- and even-number disks
within 8-raidz?
Tao
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
