Bob Friesenhahn wrote:
> On Fri, 15 Feb 2008, Luke Lonergan wrote:
>
>>> I only managed to get 200 MB/s write when I did RAID 0 across all
>>> drives using the 2540's RAID controller and with ZFS on top.
>>>
>> Ridiculously bad.
>>
>
> I agree. :-(
>
>
>>> While I agree that data is sent twice (actually up to 8X if striping
>>> across four mirrors)
>>>
>> Still only twice the data that would otherwise be sent, in other words: the
>> mirroring causes a duplicate set of data to be written.
>>
>
> Right. But more little bits of data to be sent due to ZFS striping.
>
These "little bits" should be 128kBytes by default, which should
be plenty to saturate the paths. There seems to be something else
going on here...
from the iostat data:
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
...
0.0 211.5 0.0 26940.5 1.1 33.9 5.0 160.5 99 100
c4t600A0B800039C9B500000A9C47B4522Dd0
0.0 211.5 0.0 26940.6 1.1 33.9 5.0 160.4 99 100
c4t600A0B800039C9B500000AA047B4529Bd0
0.0 154.0 0.0 19654.7 0.0 20.7 0.0 134.2 0 59
c4t600A0B80003A8A0B0000096647B453CEd0
...
shows that we have an average of 33.9 iops of 128kBytes each queued
to the storage device at a given time. There is an iop queued to the
storage device at all times (100% busy). The 59% busy device
might not always be 59% busy, but it is difficult to see from this output
because you used the "z" flag. Looks to me like ZFS is keeping the
queues full, and the device is slow to service them (asvc_t). This
is surprising, to a degree, because we would expect faster throughput
to a nonvolatile write cache.
It would be interesting to see the response for a stable idle system,
start the workload, see the fast response as we hit the write cache,
followed by the slowdown as we fill the write cache. This sort of
experiment is usually easy to create.
-- richard
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
