On Jul 9, 2009, at 4:22 AM, Jim Klimov <no-re...@opensolaris.org> wrote:
To tell the truth, I expected zvols to be faster than filesystem
datasets. They seem
to have less overhead without inodes, posix, acls and so on. So I'm
puzzled by test
results.
I'm now considering the dd i/o block size, and it means a lot
indeed, especially if
compared to zvol results with small blocks like 64k.
I ran a number of tests with a zvol recreated by commands before
each run (this
may however cause varying fragmentation impacting results of
different runs):
# zfs destroy -r pond/test; zfs create -V 30G pond/test; zfs set
compression=off pond/test; sync; dd if=/dev/zero of=/dev/zvol/rdsk/
pond/test count=1000 bs=512; sync
and tests going like
# time dd if=/dev/zero of=/dev/zvol/rdsk/pond/test count=1024 bs=1048576
1024+0 records in
1024+0 records out
real 0m42.442s
user 0m0.006s
sys 0m4.292s
The test progresses were quite jumpy (with "zpool iostat pond 1"
values varying
from 30 to 70 MBps, reads coming in sometimes).
So I'd stick to overall result - the rounded wallclock time it takes
to write 1024
records of varying size and resulting average end-user MBps. I also
write "sys"
time since that's what is consumed by the kernel and the disk
subsystem, after all.
I don't write zpool iostat speeds, since they vary too much and I
don't bother
with a spreadsheen right now. But the reported values stay about
halfway between
"wallclock MBps" ans "sys MBps" calculations, on the perceived
average, peaking
at about 350MBps for large block sizes (>4MB).
1 MB (bs=1048576): 42s (4s), 24MBps
4 MB (bs=4194304): 42s (15s), 96MBps
16MB (bs=16777216): 129s-148s (62-64s), 127-110MBps
32MB (bs=33554432, 40Gb zvol): 303s (127s), 108MBps
Similar results for writing a file to a filesystem; "zpool iostat"
values again
jumped anywhere between single MBps to GBps. Simple cleanups used
like:
# rm /pool/test30g; sync; time dd if=/dev/zero of=/pool/test30g
count=1024 bs=33554432
Values remain somewhat consistent (in the same league, at least):
1 MB (bs=1048576, 10240 blocks): 20-21s (7-8s), 512-487MBps
1 MB (bs=1048576): 2.3s (0.6s), 445MBps
4 MB (bs=4194304): 8s (3s), 512MBps
16MB (bs=16777216): 37s (15s), 442MBps
32MB (bs=33554432): 74-103s (32-42s), 442-318MBps
64Kb (bs=65536, 545664 blocks): 94s (47s), 362MBps
All in all, to make more precise results these tests should be made
in greater
numbers and averaged. But here we got some figures to think about...
On a side note, now I'll pay more attention to tuning suggestions
which involve
multi-megabyte buffers for network sockets, etc. They can actually
cause an
impact to performance many times over!
On another note,
For some reason I occasionally got results like this:
write: File too large
1+0 records in
1+0 records out
I think the zvol was not considered created by that time. In about
10-15 sec I was
able to commence the test run. Perhaps it helped that I
"initialized" the zvol by a
small write after creation, then:
# dd if=/dev/zero of=/dev/zvol/rdsk/pond/test count=1000 bs=512
Strange...
When running throughput tests the block sizes to be concerned about
are: 4k, 8k, 16k and 64k. These are the sizes that most file systems
an databases use.
If you get 4k to perform well chances are the others will fall into
line.
-Ross
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
