Without knowing more about the underlying hardware, you likely are reaching some type of IO resource constraint. Are your journals colocated or non-colocated? How fast is your backend OSD storage device?
You may also want to look at setting the norebalance flag. Good luck! > On Sep 20, 2018, at 19:52, Chen Allen <uil...@gmail.com> wrote: > > Hi there, > > Has anyone experienced below? > 2 of OSD server was down, after bring up 2 of servers, I brought 52 OSD's in > with just weight of 0.05, but it causing huge backfilling load, I saw so many > blocked requests and a number of pg stuck inactive. some of servers was > impact. so I stopped backfilling by mark nobackfill flag. everything back to > normal. > But the most strange thing happens after 2 hours, the backfilling suddenly > start again despite of nobackfill flag marked and causing so many blocked > requests then we have to reweight 52 OSD's to 0 to stabilize storage. > > Not sure why backfill start again. Anyone has any idea about that please > comments. > > Thanks so much. > Allen > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
