[ceph-users] can't start OSD

2017-11-10 Thread Rémi BUISSON
Hello, I have some issues to restart down OSDs. My cluster is running on debian stretch (with backported kernel 4.13.0) with luminous version (12.2.0). An admin changed the fsid and did restart the OSDs of one machine. I don't know if it can be the cause of all of this but my cluster is in

Re: [ceph-users] Can't Start OSD

2015-03-23 Thread Gregory Farnum
t; ? >> Could be a leveldb corruption or ceph bug. >> Now, unfortunately not enough log in that portion of the code base to >> reveal exactly why we are not getting infoos object from leveldb :-( >> >> Thanks & Regards >> Somnath >> >>

Re: [ceph-users] Can't Start OSD

2015-03-22 Thread Somnath Roy
l > Sent: Sunday, March 22, 2015 10:11 AM > To: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] Can't Start OSD > > In production for over a year, and no upgrades. > > Thanks! > > ~Noah > >> On Mar 22, 2015, at 1:01 PM, Somnath Roy wrote: >>

Re: [ceph-users] Can't Start OSD

2015-03-22 Thread Noah Mehl
bject from leveldb :-( > > Thanks & Regards > Somnath > > -Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Noah > Mehl > Sent: Sunday, March 22, 2015 10:11 AM > To: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] Can

Re: [ceph-users] Can't Start OSD

2015-03-22 Thread Somnath Roy
nath -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Noah Mehl Sent: Sunday, March 22, 2015 10:11 AM To: ceph-users@lists.ceph.com Subject: Re: [ceph-users] Can't Start OSD In production for over a year, and no upgrades. Thanks! ~Noah > On Mar

Re: [ceph-users] Can't Start OSD

2015-03-22 Thread Noah Mehl
Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Noah > Mehl > Sent: Sunday, March 22, 2015 9:34 AM > To: ceph-users@lists.ceph.com > Subject: [ceph-users] Can't Start OSD > > I have an OSD that’s failing to start. I can’t mak

Re: [ceph-users] Can't Start OSD

2015-03-22 Thread Somnath Roy
.com Subject: [ceph-users] Can't Start OSD I have an OSD that’s failing to start. I can’t make heads or tails of the error (pasted below). Thanks! ~Noah 2015-03-22 16:32:39.265116 7f4da7fa0780 0 ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7), process ceph-osd, pid 1348

[ceph-users] Can't Start OSD

2015-03-22 Thread Noah Mehl
I have an OSD that’s failing to start. I can’t make heads or tails of the error (pasted below). Thanks! ~Noah 2015-03-22 16:32:39.265116 7f4da7fa0780 0 ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7), process ceph-osd, pid 13483 2015-03-22 16:32:39.269499 7f4da7fa0780 1 files

Re: [ceph-users] Can't start osd- one osd alway be down.

2014-10-28 Thread tuantb
Hi Craig Lewis, My pool have 300TB DATA, I can't recreate a new pool, then copying data by "ceph cp pool" (take very long time). I upgraded Ceph to Giant (0.86), but still error :(( I think my proplem is "objects misplaced (0.320%)" # ceph pg 23.96 query "num_objects_missing_on_primary"

Re: [ceph-users] Can't start osd- one osd alway be down.

2014-10-27 Thread Craig Lewis
My experience is that once you hit this bug, those PGs are gone. I tried marking the primary OSD OUT, which caused this problem to move to the new primary OSD. Luckily for me, my affected PGs were using replication state in the secondary cluster. I ended up deleting the whole pool and recreating

Re: [ceph-users] Can't start osd- one osd alway be down.

2014-10-25 Thread Ta Ba Tuan
#ceph pg *6.9d8* query ... "peer_info": [ { "peer": "49", "pgid": "6.9d8", "last_update": "102889'7801917", "last_complete": "102889'7801917", "log_tail": "102377'7792649", "last_user_version": 7801879, "last_backfill": "MAX",

Re: [ceph-users] Can't start osd- one osd alway be down.

2014-10-25 Thread Ta Ba Tuan
My Ceph was hung, and"osd.21 172.30.5.2:6870/8047 879 : [ERR] 6.9d8 has 4 objects unfound and apparently lost". After I restart all ceph-data nodes, I can't start osd.21, have many logs about pg 6.9d8 as: -440> 2014-10-25 19:28:17.468161 7fec5731d700 5 -- op tracker -- seq: 3083, time

Re: [ceph-users] Can't start osd- one osd alway be down.

2014-10-25 Thread Ta Ba Tuan
I send some related bugs: (osd.21 not be able started) -8705> 2014-10-25 14:41:04.345727 7f12bac2f700 5 *osd.21* pg_epoch: 102843 pg[*6.5e1*( v 102843'11832159 (102377'11822991,102843'11832159] lb c4951de1/rbd_data.3955c5cdbb2ea.000405f0/head//6 local-les=101780 n=4719 ec=164 les/c 1

Re: [ceph-users] Can't start osd- one osd alway be down.

2014-10-24 Thread Ta Ba Tuan
Hi Craig, Thanks for replying. When i started that osd, Ceph Log from "ceph -w" warns pgs 7.9d8 23.596, 23.9c6, 23.63 can't recovery as pasted log. Those pgs are "active+degraded" state. #ceph pg map 7.9d8 osdmap e102808 pg 7.9d8 (7.9d8) -> up [93,49] acting [93,49] (When start osd.21 then pg

Re: [ceph-users] Can't start osd- one osd alway be down.

2014-10-24 Thread Craig Lewis
It looks like you're running into http://tracker.ceph.com/issues/5699 You're running 0.80.7, which has a fix for that bug. From my reading of the code, I believe the fix only prevents the issue from occurring. It doesn't work around or repair bad snapshots created on older versions of Ceph. Wer

[ceph-users] Can't start osd- one osd alway be down.

2014-10-23 Thread Ta Ba Tuan
Dear everyone I can't start osd.21, (attached log file). some pgs can't be repair. I'm using replicate 3 for my data pool. Feel some objects in those pgs be failed, I tried to delete some data that related above objects, but still not start osd.21 and, removed osd.21, but other osds (eg: osd.8

Re: [ceph-users] Can't start OSD

2014-08-08 Thread Matt Harlum
eph-10) mount failed to open journal > /var/lib/ceph/osd/ceph-10/journal: (2) No such file or directory > 2014-08-08 08:49:20.146797 7f2c40c1a7a0 -1 ** ERROR: error converting store > /var/lib/ceph/osd/ceph-10: (2) No such file or directory > > From: German Anders [mailto:gand..

Re: [ceph-users] Can't start OSD

2014-08-08 Thread O'Reilly, Dan
7a0 -1 ** ERROR: error converting store /var/lib/ceph/osd/ceph-10: (2) No such file or directory From: German Anders [mailto:gand...@despegar.com] Sent: Friday, August 08, 2014 8:23 AM To: O'Reilly, Dan Cc: Karan Singh; ceph-users@lists.ceph.com Subject: Re: [ceph-users] Can't start OSD

Re: [ceph-users] Can't start OSD

2014-08-08 Thread German Anders
How about the logs? Is something there? ls /var/log/ceph/ German Anders --- Original message --- Asunto: Re: [ceph-users] Can't start OSD De: "O'Reilly, Dan" Para: Karan Singh Cc: ceph-users@lists.ceph.com Fecha: Friday, 08/08/2014 10:53 Nope. Not

Re: [ceph-users] Can't start OSD

2014-08-08 Thread O'Reilly, Dan
osd.19 down0 I'm missing something here. I don't know if it's a config issue or what. But the docs haven't helped me. From: Karan Singh [mailto:karan.si...@csc.fi] Sent: Friday, August 08, 2014 1:11 AM To: O'Reilly, Dan Cc: ceph-users@lists.ceph.com Subjec

Re: [ceph-users] Can't start OSD

2014-08-08 Thread Karan Singh
Try to make these OSD IN ceph osd in osd.12 osd.13 osd.14 osd.15 Then restart osd services - Karan Singh - On 08 Aug 2014, at 00:55, O'Reilly, Dan wrote: > # idweight type name up/down reweight > -1 7.2 root default > -2 1.8 host tm1cldosdl01 > 0 0

[ceph-users] Can't start OSD

2014-08-07 Thread O'Reilly, Dan
# idweight type name up/down reweight -1 7.2 root default -2 1.8 host tm1cldosdl01 0 0.45osd.0 up 1 1 0.45osd.1 up 1 2 0.45osd.2 up 1 3 0.45