2012-05-16 22:21, bofh wrote:
There's something going on then. I have 7x 3TB disk at home, in raidz3, so about 12TB usable. 2.5TB actually used. Scrubbing takes about 2.5 hours. I had done the resilvering as well, and that did not take 15 hours/drive.
That is the critical moment ;) The system we plan to upgrade runs nearly full for a couple of years now, with some overflowing data spilling out to another server or performant workstation. This system has a lot of writes and rewrites (rolling backups, VM images, home dirs and compilations, document storage and so on) compounded by zfs-auto-snap services. Due to the fact that there are many rewrites like this, fragmentation is a substantial hit to scrub and resilver performance which both need to walk the whole block-pointer tree which yields lots of random reads of small blocks from all over the pool. I began writing a letter with guesses/questions about that, but it seems to be lengthy and may get published in a few days ;)
Additionally, if you're only replacing one drive at a time, you're only resilvering 250GB at a time, regardless of the size of the new drive.
That is true.
If you already have 45X 3TB drives waiting to go in, bite the bullet and get that eSATA cage, since you want to re-do your zpools. You can reuse it for offsite backups in the future.
That is a good plan, except that the server's owners plan to upgrade a dozen drives for now. Even this would triple the current pool's size in a quarter of disk bays used. They only have one for testing on hands and plan to buy a dozen more; there are no 45 drives waiting. There is no spoon, either ;) I will try to get them into making a (local/offsite) backup box as well. Since a new server (from x4540 to a handmade Supermicro) would likely be more performant and have more RAM, I expect that this Thumper would be the backup box for a new server, ultimately. Thanks, //Jim _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
