Hello all,
I had a hard reset on a ceph node, and one of the OSDs is not starting
due to leveldb error. At that moment, the node was trying to start up,
but there was no actual writing of new data:
2016-01-27 12:00:37.068431 7f367f654880 0 ceph version 0.94.5
(9764da52395923e0b32908d83a9f7304401fee43), process ceph-osd, pid 24734
2016-01-27 12:00:37.115800 7f367f654880 0
filestore(/var/lib/ceph/osd/ceph-26) backend xfs (magic 0x58465342)
2016-01-27 12:00:37.133031 7f367f654880 0
genericfilestorebackend(/var/lib/ceph/osd/ceph-26) detect_features:
FIEMAP ioctl is supported and appears to work
2016-01-27 12:00:37.133042 7f367f654880 0
genericfilestorebackend(/var/lib/ceph/osd/ceph-26) detect_features:
FIEMAP ioctl is disabled via 'filestore fiemap' config option
2016-01-27 12:00:37.136538 7f367f654880 0
genericfilestorebackend(/var/lib/ceph/osd/ceph-26) detect_features:
syncfs(2) syscall fully supported (by glibc and kernel)
2016-01-27 12:00:37.137584 7f367f654880 0
xfsfilestorebackend(/var/lib/ceph/osd/ceph-26) detect_feature: extsize
is supported and kernel 3.10.0-123.el7.x86_64 >= 3.5
2016-01-27 12:00:37.176226 7f367f654880 -1
filestore(/var/lib/ceph/osd/ceph-26) Error initializing leveldb :
Corruption: 1 missing files; e.g.:
/var/lib/ceph/osd/ceph-26/current/omap/075074.sst
2016-01-27 12:00:37.176286 7f367f654880 -1 osd.26 0 OSD:init: unable to
mount object store
2016-01-27 12:00:37.176315 7f367f654880 -1 ** ERROR: osd init failed:
(1) Operation not permitted
The file 075074.sst is missing indeed.
Since I was not able to restart the OSD, and I could not find
information on recovering the leveldb, I marked the OSD as lost, but
then I got 3 incomplete OSDs. I tried to follow the recovery howto at
https://ceph.com/community/incomplete-pgs-oh-my/, but it stepped on the
same error on leveldb, having the same dependency.
Is there any means to recover from this situation? To check and recover
leveldb as good as possible? Or, alternatively, to get rid of the
incomplete status, even with the penalty of losing some objects?
Regards.
--
As informações contidas nesta mensagem são CONFIDENCIAIS, protegidas pelo
sigilo legal e por direitos autorais. A divulgação, distribuição, reprodução ou
qualquer forma de utilização do teor deste documento depende de autorização do
emissor, sujeitando-se o infrator às sanções legais. Caso esta comunicação
tenha sido recebida por engano, favor avisar imediatamente, respondendo esta
mensagem.
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
