Hi Caspar, On Thu, Jan 10, 2019 at 1:31 PM Caspar Smit <caspars...@supernas.eu> wrote: > > Hi all, > > I wanted to test Dan's upmap-remapped script for adding new osd's to a > cluster. (Then letting the balancer gradually move pgs to the new OSD > afterwards)
Cool. Insert "no guarantees or warranties" comment here. And btw, I have noticed that the method doesn't always work in this use case: sometimes the upmap balancer will not remove pg-upmap-items entries if there are very few severely underloaded OSDs. The calc_pg_upmaps might need some code changes to fully work in this scenario. > I've created a fresh (virtual) 12.2.10 4-node cluster with very small disks > (16GB each). 2 OSD's per node. > Put ~20GB of data on the cluster. > > Now when i set the norebalance flag and add a new OSD, 99% of pgs end up > recovering or in recovery_wait. Only a few will be backfill_wait. > > The recovery starts as expected (norebalance only stops backfilling pgs) and > finished eventually > > The upmap-remapped script only works with pgs which need to be backfilled. The script looks for pgs which are remapped but not degraded, then uses pg-upmap-items to move pgs from the state active+remapped to active+clean. > it does work for the handful of pgs in backfill_wait status but my question > is: > > When is ceph doing recovery in stead of backfilling? Only when the cluster is > rather empty or what is the criteria? Are the OSD's too small? Vaguely, recovery is done when the pg log holds all of the changes made on a pg while the pg was degraded. Once that pg log was overfilled, the osd needs to backfill. In other words: recovery is a process to replay a log of ops. But that log has a size limit, so backfilling is the fallback to scan all objects for changes. Cheers, Dan > > Kind regards, > Caspar > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
