Hi Bryan, How does the norebalance procedure work? You set the flag, increase the weight, then I expect the PGs to stay in remapped unless they're degraded ... why would a PG be degraded just because of a weight change? And then what happens when you unset norebalance?
Cheers, Dan On Mon, Aug 7, 2017 at 6:07 PM, Bryan Stillwell <bstillw...@godaddy.com> wrote: > Dan, > > We recently went through an expansion of an RGW cluster and found that we > needed 'norebalance' set whenever making CRUSH weight changes to avoid slow > requests. We were also increasing the CRUSH weight by 1.0 each time which > seemed to reduce the extra data movement we were seeing with smaller weight > increases. Maybe something to try out next time? > > Bryan > > From: ceph-users <ceph-users-boun...@lists.ceph.com> on behalf of Dan van der > Ster <d...@vanderster.com> > Date: Friday, August 4, 2017 at 1:59 AM > To: Laszlo Budai <las...@componentsoft.eu> > Cc: ceph-users <ceph-users@lists.ceph.com> > Subject: Re: [ceph-users] expanding cluster with minimal impact > > Hi Laszlo, > > The script defaults are what we used to do a large intervention (the > default delta weight is 0.01). For our clusters going any faster > becomes disruptive, but this really depends on your cluster size and > activity. > > BTW, in case it wasn't clear, to use this script for adding capacity > you need to create the new OSDs to your cluster with initial crush > weight = 0.0 > > osd crush initial weight = 0 > osd crush update on start = true > > -- Dan > > > > On Thu, Aug 3, 2017 at 8:12 PM, Laszlo Budai <las...@componentsoft.eu> wrote: > Dear all, > > I need to expand a ceph cluster with minimal impact. Reading previous > threads on this topic from the list I've found the ceph-gentle-reweight > script > (https://github.com/cernceph/ceph-scripts/blob/master/tools/ceph-gentle-reweight) > created by Dan van der Ster (Thank you Dan for sharing the script with us!). > > I've done some experiments, and it looks promising, but it is needed to > properly set the parameters. Did any of you tested this script before? what > is the recommended delta_weight to be used? From the default parameters of > the script I can see that the default delta weight is .5% of the target > weight that means 200 reweighting cycles. I have experimented with a > reweight ratio of 5% while running a fio test on a client. The results were > OK (I mean no slow requests), but my test cluster was a very small one. > > If any of you has done some larger experiments with this script I would be > really interested to read about your results. > > Thank you! > Laszlo > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
