I've been playing with re-adding the bricks and here is some interesting behavior.
When I try to force add the bricks to the volume while it's running, I get complaints about one of the bricks already being a member of a volume. If I stop the volume, I can then force-add the bricks. However, the volume won't start without force. Once the volume is force started, all of the bricks remain offline. I feel like I'm close...but not quite there... > From: "anthony" <[email protected]> > To: "Strahil Nikolov" <[email protected]> > Cc: "gluster-users" <[email protected]> > Sent: Tuesday, September 7, 2021 7:45:44 AM > Subject: Re: [Gluster-users] Recovering from remove-brick where shards did not > rebalance > I was contemplating these options, actually, but not finding anything in my > research showing someone had tried either before gave me pause. > One thing I wasn't sure about when doing a force add-brick was if gluster > would > wipe the existing data from the added bricks. Sounds like that may not be the > case? > With regards to concatenating the main file + shards, how would I go about > identifying the shards that pair with the main file? I see the shards have > sequence numbers, but I'm not sure how to match the identifier to the main > file. > Thanks!! >> From: "Strahil Nikolov" <[email protected]> >> To: "anthony" <[email protected]>, "gluster-users" <[email protected]> >> Sent: Tuesday, September 7, 2021 6:02:36 AM >> Subject: Re: [Gluster-users] Recovering from remove-brick where shards did >> not >> rebalance >> The data should be recoverable by concatenating the main file with all >> shards. >> Then you can copy the data back via the FUSE mount point. >> I think that some users reported that add-brick with the force option allows >> to >> 'undo' the situation and 're-add' the data, but I have never tried that and I >> cannot guarantee that it will even work. >> The simplest way is to recover from a recent backup , but sometimes this >> leads >> to a data loss. >> Best Regards, >> Strahil Nikolov >>> On Tue, Sep 7, 2021 at 9:29, Anthony Hoppe >>> <[email protected]> wrote: >>> Hello, >>> I did a bad thing and did a remove-brick on a set of bricks in a >>> distributed-replicate volume where rebalancing did not successfully >>> rebalance >>> all files. In sleuthing around the various bricks on the 3 node pool, it >>> appears that a number of the files within the volume may have been stored as >>> shards. With that, I'm unsure how to proceed with recovery. >>> Is it possible to re-add the removed bricks somehow and then do a heal? Or >>> is >>> there a way to recover data from shards somehow? >>> Thanks! >>> ________ >>> Community Meeting Calendar: >>> Schedule - >>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC >>> Bridge: [ https://meet.google.com/cpu-eiue-hvk | >>> https://meet.google.com/cpu-eiue-hvk ] >>> Gluster-users mailing list >>> [ mailto:[email protected] | [email protected] ] >>> [ https://lists.gluster.org/mailman/listinfo/gluster-users | >>> https://lists.gluster.org/mailman/listinfo/gluster-users ]
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-users
