On Wed, Mar 21, 2012 at 9:51 AM, Jim Klimov <jimkli...@cos.ru> wrote:
> 2012-03-21 17:28, Edward Ned Harvey wrote:
>> It's not advisable to put more than ~8 disks in a single vdev, because it
>> really hurts during resilver time. Maybe a week or two to resilver like
>> that.
>
> Yes, that's important to note also. While ZFS marketing initially
> stressed that unlike traditional RAID systems, a "rebuild" of ZFS
> onto a spare/replacement disk only needs to copy referenced data
> and not the whole disk, it somehow fell off the picture that such
> rebuild is a lot of random IO - because the data block tree must
> be read in as a tree walk, often with emphasis on block "age" (its
> birth TXG number). If your pool is reasonably full (and who runs
> it empty?) then this is indeed lots of random IO, and a blind
> full-disk copy would have gone orders of magnitude faster.
> The less disk participate in this thrashing - the faster it will
> go (less data needed overall to reconstruct a disk's worth of
> sectors from redundancy data).
Three are two different cases here... resilver to reconstruct
data from a failed drive and a scrub to pro-actively find bad sectors.
The best case situation for the first case (bad drive
replacement) is a mirrored drive in my experience. In that case only
the data involved in the failure needs to be read and written. I am
unclear how much of the data is read in the case of a failure of a
drive in a RAIDz<n> vdev _from_other_vdevs_. I have seen disk activity
on non-failure related vdevs during a drive replacement, which is why
I am unsure in this case.
In the case of a "scrub", _all_ of the data in the zpool is read
and the checksums checked. My 22 vdev zpool takes about 300 hours for
this while the 2 vdev zpool takes over 600 hours. Both have comparable
amounts of data and snapshots. The 22 vdev zpool is on a production
server with normal I/O activity, the 2 vdev case is only receiving zfs
snapshots and doing no other I/O.
--
{--------1---------2---------3---------4---------5---------6---------7---------}
Paul Kraus
-> Senior Systems Architect, Garnet River ( http://www.garnetriver.com/ )
-> Sound Coordinator, Schenectady Light Opera Company (
http://www.sloctheater.org/ )
-> Technical Advisor, RPI Players
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
