Well...i can only say "well said". BTW i have a raidz2 with 9 vdevs with 4 disks each (sata enterprise disks) and the scrub of the pool takes between 12 to 39 hours..depends on the workload of the server. So far it's acceptable but each case is a case i think...
Bruno On 16-3-2010 14:04, Khyron wrote: > In following this discussion, I get the feeling that you and Richard > are somewhat > talking past each other. He asked you about the hardware you are > currently running > on, whereas you seem to be interested in a model for the impact of > scrubbing on > I/O throughput that you can apply to some not-yet-acquired hardware. > > It should be clear by now that the model you are looking for does not > exist given > how new ZFS is, and Richard has been focusing his comments on your > existing (home) > configuration since that is what you provided specs for. > > Since you haven't provided specs for this larger system you may be > purchasing in the > future, I don't think anyone can give you specific guidance on what > the I/O impact of > scrubs on your configuration will be. Richard seems to be giving more > design guidelines > and hints, and just generally good to know information to keep in mind > while designing > your solution. Of course, he's been giving it in the context of your > 11 disk wide > RAIDZ2 and not the 200 TB monster you only described in the last e-mail. > > Stepping back, it may be worthwhile to examine the advice Richard has > given, in the > context of the larger configuration. > > First, you won't be using commodity hardware for your enterprise-class > storage system, > will you? > > Second, I would imagine that as a matter of practice, most people > schedule their pools > to scrub as far away from prime hours as possible. Maybe it's > possible, and maybe it's > not. The question to the larger community should be "who is running a > 100+ TB pool > and how have you configured your scrubs?" Or even "for those running > 100+ TB pools, > do your scrubs interfere with your production traffic/throughput? If > so, how do you > compensate for this?" > > Third, as for ZFS scrub prioritization, Richard answered your question > about that. He > said it is low priority and can be tuned lower. However, he was > answering within the > context of an 11 disk RAIDZ2 with slow disks His exact words were: > > "This could be tuned lower, but your storage is slow and *any* I/O > activity will be > noticed." > > If you had asked about a 200 TB enterprise-class pool, he may have had > a different > response. I don't know if ZFS will make different decisisons > regarding I/O priority on > commodity hardware as opposed to enterprise hardware, but I imagine it > does *not*. > If I'm mistaken, someone should correct me. Richard also said "In > b133, the priority > scheduler will work better than on older releases." That may not be > an issue since > you haven't acquired your hardware YET, but again, Richard didn't know > that you > were talking about a 200 TB behemoth because you never said that. > > Fourth, Richard mentioned a wide RAIDZ2 set. Hopefully, if nothing > else, we've > seen that designing larger ZFS storage systems with pools composed of > smaller top > level VDEVs works better, and preferably mirrored top level VDEVs in > the case of lots > of small, random reads. You didn't indicate the profile of the data > to be stored on > your system, so no one can realistically speak to that. I think the > general guidance > is sound. Multiple top level VDEVs, preferably mirrors. If you're > creating RAIDZ2 > top level VDEVs, then they should be smaller (narrower) in terms of > the number of > disks in the set. 11 would be too many, based on what I have seen and > heard on > this list cross referenced with the (little) information you have > provided. > > RAIDZ2 would appear to require more CPU power that RAIDZ, based on the > report > you gave and thus may have less negative impact on the performance of > your storage > system. I'll cop to that. However, you never mentioned how your 200 > TB behemoth > system will be used, besides an off-hand remark about CIFS. Will it > be serving CIFS? > NFS? Raw ZVOLs over iSCSI? You never mentioned any of that. Asking > about CIFS > if you're not going to serve CIFS doesn't make much sense. That would > appear to > be another question for the ZFS gurus here -- WHY does RAIDZ2 cause so > much > negative performance impact on your CIFS service while RAIDZ does > not? Your > experience is that a scrub of a RAIDZ2 maxed CPU while a RAIDZ scrub > did not, right? > > Fifth, the pool scrub should probably be as far away from peak usage > times as possible. > That may or may not be feasible, but I don't think anyone would > disagree with that > advice. Again, I know there are people running large pools who > perform scrubs. It > might be worthwhile to directly ask what these people have experienced > in terms of > scrub performance on RAIDZ vs. RAIDZ2, or in general. > > Finally, I would also note that Richard has been very responsive to > your questions (in > his own way) but you increasingly seem to be hostile and even > disrespectful toward > him. (I've noticed this in more then one of your e-mails; they sound > progressively > more self-centered and selfish. That's just my opinion.) If this is > a community, that's > not a helpful way to treat a senior member of the community, even if > he's not > answering the question you want answered. > > Keep in mind that asking the wrong questions is the leading cause of > wrong answers, > as a former boss of mine likes to say. Maybe you would have gotten > responses you > found more useful and less insulting had you phrased your questions in > an different > way? > > And no, Richard doesn't need me to defend him, especially since I > don't know him > from a can of paint. Your attacks (for lack of a better word) on him > seem > unwarranted and I can't stay silent about it any longer. > > Anyway, hopefully that helps in some way. And hopefully you'll get > how you're > appearing to others who are reading your words. Right now, in MY > opinion alone, > you look like a n00b who isn't respectful enough to deserve the help > of anyone > here. > > On Tue, Mar 16, 2010 at 07:35, Tonmaus <sequoiamo...@gmx.net > <mailto:sequoiamo...@gmx.net>> wrote: > > Hi Richard, > > > > - scrubbing the same pool, configured as raidz1 > > didn't max out CPU which is no surprise (haha, slow > > storage...) the notable part is that it didn't slow > > down payload that much either. > > > > raidz creates more, smaller writes than a mirror or > > simple stripe. If the disks are slow, > > then the IOPS will be lower and the scrub takes > > longer, but the I/O scheduler can > > manage the queue better (disks are slower). > > This wasn't mirror vs. raidz but raidz1 vs. raidz2, whereas the > latter maxes out CPU and the former maxes out physical disc I/O. > Concurrent payload degradation isn't that extreme on raidz1 pools, > as it seems. Hence, the CPU theory that you still seem to be > reluctant to follow. > > > > There are several > > bugs/RFEs along these lines, something like: > > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bu > > g_id=6743992 > > Thanks to pointing at this. As it seems, it's a problem for a > couple of years already. Obviously, the opinion is being shared > that this a management problem, not a HW issue. > > As a Project Manager I will soon have to take a purchase decision > for an archival storage system (A/V media), and one of the options > we are looking into is SAMFS/QFS solution including tiers on disk > with ZFS. I will have to make up my mind if the pool sizes we are > looking into (typically we will need 150-200 TB) are really > manageable under the current circumstances when we think about > including zfs scrub in the picture. From what I have learned here > it rather looks as if there will be an extra challenge, if not > even a problem for the system integrator. That's unfortunate. > > Regards, > > Tonmaus > -- > This message posted from opensolaris.org <http://opensolaris.org> > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org <mailto:zfs-discuss@opensolaris.org> > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > > > > -- > "You can choose your friends, you can choose the deals." - Equity Private > > "If Linux is faster, it's a Solaris bug." - Phil Harman > > Blog - http://whatderass.blogspot.com/ > Twitter - @khyron4eva > > > -- > This message has been scanned for viruses and > dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is > believed to be clean. > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
