Hi Vladimir, While it is advisable to investigate why deep-scrub is killing your performance (it's enabled for a reason) and find ways to fix that (seperate block.db SSD's for instance might help) here's a way to accomodate your needs:
For all your 7200RPM Spinner based pools do: ceph osd pool set <poolname> no-scrub true ceph osd pool set <poolname> nodeep-scrub true Though, you might want to leave normal scrubbing on and see if that is enough. Normal scrubbing has way less impact on performance then deep-scrub. When you've set all your spinner pools with above flag(s) you can unset the global no-scrub and nodeep-scrub flags and your health warning goes away (and scrubbing does not occur on those pools). Kind regards, Caspar Op ma 10 dec. 2018 om 12:06 schreef Vladimir Prokofev <v...@prokofev.me>: > Hello list. > > Deep scrub totally kills cluster performance. > First of all, it takes several minutes to complete: > 2018-12-09 01:39:53.857994 7f2d32fde700 0 log_channel(cluster) log [DBG] > : 4.75 deep-scrub starts > 2018-12-09 01:46:30.703473 7f2d32fde700 0 log_channel(cluster) log [DBG] > : 4.75 deep-scrub ok > > Second, while it runs, it consumes 100% of OSD time[1]. This is on an > ordinary 7200RPM spinner. > While this happens, VMs cannot access their disks, and that leads to > service interruptions. > > I disabled scrub and deep-scrub operations for now, and have 2 major > questions: > - can I disable 'health warning' status for noscrub and nodeep-scrub? I > thought there was a way to do this, but can't find it. I want my cluster to > think it's healthy, so if any new 'slow requests' or anything else pops - > it will change status to 'health warning' again; > - is there a way to limit deepscrub impact on disk performance, or do I > just have to go and buy SSDs? > > [1] https://imgur.com/a/TKH3uda > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
