On Mar 14, 2010, at 11:25 PM, Tonmaus wrote: > Hello again, > > I am still concerned if my points are being well taken. > >> If you are concerned that a >> single 200TB pool would take a long >> time to scrub, then use more pools and scrub in >> parallel. > > The main concern is not scrub time. Scrub time could be weeks if scrub just > would behave. You may imagine that there are applications where segmentation > is a pain point, too.
I agree. >> The scrub will queue no more >> han 10 I/Os at one time to a device, so devices which >> can handle concurrent I/O >> are not consumed entirely by scrub I/O. This could be >> tuned lower, but your storage >> is slow and *any* I/O activity will be noticed. > > There are a couple of things I maybe don't understand, then. > > - zpool iostat is reporting more than 1k of outputs while scrub ok > - throughput is as high as can be until maxing out CPU You would rather your CPU be idle? What use is an idle CPU, besides wasting energy :-)? > - nominal I/O capacity of a single device is still around 90, how can 10 I/Os > already bring down payload 90 IOPS is approximately the worst-case rate for a 7,200 rpm disk for a small, random workload. ZFS tends to write sequentially, so "random writes" tend to become "sequential writes" on ZFS. So it is quite common to see scrub workloads with >> 90 IOPS. > - scrubbing the same pool, configured as raidz1 didn't max out CPU which is > no surprise (haha, slow storage...) the notable part is that it didn't slow > down payload that much either. raidz creates more, smaller writes than a mirror or simple stripe. If the disks are slow, then the IOPS will be lower and the scrub takes longer, but the I/O scheduler can manage the queue better (disks are slower). > - scrub is obviously fine with data added or deleted during a pass. So, it > could be possible to pause and resume a pass, couldn't it? You can start or stop scrubs, there no resume directive. There are several bugs/RFEs along these lines, something like: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6743992 > My conclusion from these observations is that not only disk speed counts > here, but other bottlenecks may strike as well. Solving the issue by the > wallet is one way, solving it by configuration of parameters is another. So, > is there a lever for scrub I/O prio, or not? Is there a possibility to pause > scrub passed and resume? Scrub is already the lowest priority. Would you like it to be lower? I think the issue is more related to which queue is being managed by the ZFS priority scheduler rather than the lack of scheduling priority. -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Atlanta, March 16-18, 2010 http://nexenta-atlanta.eventbrite.com Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
