The balancer is the usual way of using a remapper. * Set norebalance * Create the OSDs * Run the remapper so the backfill goes away * Unset norebalance * The balancer then will incrementally remove the extra upmaps, gradually evening out data.
What did you do? Did you compute a set of upmaps by hand and apply them? > On Jun 21, 2025, at 11:00 AM, Michal Strnad <michal.str...@cesnet.cz> wrote: > > Hi. > > Thank you for your reply > > The balancer is off, so it shouldn't be interfering with that. Do you happen > to have a list of steps (maybe commands from the history) on how you launched > it? We're trying to figure out what we're missing. > > Regards, > Michal > > > On 6/21/25 15:21, Wesley Dillingham wrote: >> Michal; >> Is the balancer (upmap balancer) running (ceph balancer status) and undoing >> some of the upmaps the pgremapper is introducing? >> Respectfully, >> *Wes Dillingham* >> LinkedIn <http://www.linkedin.com/in/wesleydillingham> >> w...@wesdillingham.com >> On Sat, Jun 21, 2025 at 2:05 AM Michal Strnad <michal.str...@cesnet.cz> >> wrote: >>> Hi. >>> >>> Over the past few days, we've been working on migrating data from >>> machine A to machine B using the pgremapper tool, but we haven’t been >>> able to achieve the expected results. >>> >>> As part of our testing, we set up a small Ceph cluster with several >>> monitors, managers, and servers with OSDs.We applied the flags noout, >>> nobackfill, norecovery, and norebalance, and then added additional >>> servers with OSDs. While Ceph did allocate PG replicas to the newly >>> added OSDs, the actual data didn’t move due to the active flags. We then >>> attempted to use pgremapper to migrate all PGs from one server to the >>> new one, removing or negating the flags in the process. However, we >>> frequently failed to complete the migration of all data/PGs. >>> Are we overlooking something? Does anyone have a reliable, step-by-step >>> procedure we can follow to perform this correctly? >>> >>> Any help would be greatly appreciated! >>> >>> Michal >>> >>> >>> On 3/19/25 08:13, Janne Johansson wrote: >>>>> The safest approach would be to use the upmap-remapped.py tool >>> developed by Dan at CERN. See [1] for details. >>>>> >>>>> The idea is to leverage the upmap load balancer to progressively >>> migrate the data to the new servers, minimizing performance impact on the >>> cluster and clients. I like to create the OSDs ahead of time on the nodes >>> that I initially place in a root directory called ‘closet’. >>>>> >>>>> I then apply the norebalance flag (ceph osd set norebalance), disable >>> the balancer (ceph balancer off), move the new nodes with already >>> provisioned OSDs to their final location (rack), run ./upmap-remapped.py to >>> bring all PGs back to active+clean state, remove the norebalance flag (ceph >>> osd unset norebalance), re-enable the balancer (ceph balancer on) and watch >>> data moving progressively as the upmap balancer executes its plans. >>>> >>>> We do exactly that also, sometimes using pgremapper instead of >>>> upmap-remapper.py, but the effect is the same. Make the changes with >>>> norebalance, upmap the PGs to be happy where they are until we unset >>>> norebalance and let the ceph balancer correct it X% at a time. >>>> >>> >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-le...@ceph.io >>> > > -- > Michal Strnad > Storage specialist > CESNET a.l.e. > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
