That will be down to the pool the rbd was in, the crush rule for that pool will dictate which osd's store objects. In a standard config that rbd will likely have objects on every osd in your cluster.
On 8 Aug 2016 9:51 a.m., "Georgios Dimitrakakis" <gior...@acmac.uoc.gr> wrote: > Hi, >> >> >> On 08.08.2016 09:58, Georgios Dimitrakakis wrote: >> >>> Dear all, >>> >>> I would like your help with an emergency issue but first let me describe >>> our environment. >>> >>> Our environment consists of 2OSD nodes with 10x 2TB HDDs each and 3MON >>> nodes (2 of them are the OSD nodes as well) all with ceph version 0.80.9 >>> (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047) >>> >>> This environment provides RBD volumes to an OpenStack Icehouse >>> installation. >>> >>> Although not a state of the art environment is working well and within >>> our expectations. >>> >>> The issue now is that one of our users accidentally deleted one of the >>> volumes without keeping its data first! >>> >>> Is there any way (since the data are considered critical and very >>> important) to recover them from CEPH? >>> >> >> Short answer: no >> >> Long answer: no, but.... >> >> Consider the way Ceph stores data... each RBD is striped into chunks >> (RADOS objects with 4MB size by default); the chunks are distributed >> among the OSDs with the configured number of replicates (probably two >> in your case since you use 2 OSD hosts). RBD uses thin provisioning, >> so chunks are allocated upon first write access. >> If an RBD is deleted all of its chunks are deleted on the >> corresponding OSDs. If you want to recover a deleted RBD, you need to >> recover all individual chunks. Whether this is possible depends on >> your filesystem and whether the space of a former chunk is already >> assigned to other RADOS objects. The RADOS object names are composed >> of the RBD name and the offset position of the chunk, so if an >> undelete mechanism exists for the OSDs' filesystem, you have to be >> able to recover file by their filename, otherwise you might end up >> mixing the content of various deleted RBDs. Due to the thin >> provisioning there might be some chunks missing (e.g. never allocated >> before). >> >> Given the fact that >> - you probably use XFS on the OSDs since it is the preferred >> filesystem for OSDs (there is RDR-XFS, but I've never had to use it) >> - you would need to stop the complete ceph cluster (recovery tools do >> not work on mounted filesystems) >> - your cluster has been in use after the RBD was deleted and thus >> parts of its former space might already have been overwritten >> (replication might help you here, since there are two OSDs to try) >> - XFS undelete does not work well on fragmented files (and OSDs tend >> to introduce fragmentation...) >> >> the answer is no, since it might not be feasible and the chance of >> success are way too low. >> >> If you want to spend time on it I would propose the stop the ceph >> cluster as soon as possible, create copies of all involved OSDs, start >> the cluster again and attempt the recovery on the copies. >> >> Regards, >> Burkhard >> > > Hi! Thanks for the info...I understand that this is a very difficult and > probably not feasible task but in case I need to try a recovery what other > info should I need? Can I somehow find out on which OSDs the specific data > were stored and minimize my search there? > Any ideas on how should I proceed? > > > Best, > > G. > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
