Re: [ceph-users] OSD turned itself off

2015-02-16 Thread Josef Johansson
And yeah, it’s the same EIO 5 error. So ok, the errors doesn’t show anything useful to the osd crash. > On 16 Feb 2015, at 21:58, Josef Johansson wrote: > > Well, I knew it had all the correct information since earlier so gave it a > shot :) > > Anyway, I think it may be just a bad controlle

Re: [ceph-users] OSD turned itself off

2015-02-16 Thread Josef Johansson
Well, I knew it had all the correct information since earlier so gave it a shot :) Anyway, I think it may be just a bad controller as well. New enterprise drives shouldn’t be giving read errors this early in deployment tbh. Cheers, Josef > On 16 Feb 2015, at 17:37, Greg Farnum wrote: > > Woah

Re: [ceph-users] OSD turned itself off

2015-02-16 Thread Greg Farnum
Woah, major thread necromancy! :) On Feb 13, 2015, at 3:03 PM, Josef Johansson wrote: > > Hi, > > I skimmed the logs again, as we’ve had more of this kinda errors, > > I saw a lot of lossy connections errors, > -2567> 2014-11-24 11:49:40.028755 7f6d49367700 0 -- 10.168.7.23:6819/10217 > >> 1

Re: [ceph-users] OSD turned itself off

2015-02-13 Thread Josef Johansson
Hi, I skimmed the logs again, as we’ve had more of this kinda errors, I saw a lot of lossy connections errors, -2567> 2014-11-24 11:49:40.028755 7f6d49367700 0 -- 10.168.7.23:6819/10217 >> 10.168.7.54:0/1011446 pipe(0x19321b80 sd=44 :6819 s=0 pgs=0 cs=0 l=1 c=0x110d2b00).accept replacing exis

Re: [ceph-users] OSD turned itself off

2014-06-13 Thread Josef Johansson
Thanks for the quick response. Cheers, Josef Gregory Farnum skrev 2014-06-14 02:36: On Fri, Jun 13, 2014 at 5:25 PM, Josef Johansson wrote: Hi Greg, Thanks for the clarification. I believe the OSD was in the middle of a deep scrub (sorry for not mentioning this straight away), so then it cou

Re: [ceph-users] OSD turned itself off

2014-06-13 Thread Gregory Farnum
On Fri, Jun 13, 2014 at 5:25 PM, Josef Johansson wrote: > Hi Greg, > > Thanks for the clarification. I believe the OSD was in the middle of a deep > scrub (sorry for not mentioning this straight away), so then it could've > been a silent error that got wind during scrub? Yeah. > > What's best pr

Re: [ceph-users] OSD turned itself off

2014-06-13 Thread Josef Johansson
Hi Greg, Thanks for the clarification. I believe the OSD was in the middle of a deep scrub (sorry for not mentioning this straight away), so then it could've been a silent error that got wind during scrub? What's best practice when the store is corrupted like this? Cheers, Josef Gregory Far

Re: [ceph-users] OSD turned itself off

2014-06-13 Thread Gregory Farnum
The OSD did a read off of the local filesystem and it got back the EIO error code. That means the store got corrupted or something, so it killed itself to avoid spreading bad data to the rest of the cluster. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Fri, Jun 13, 2014 a

[ceph-users] OSD turned itself off

2014-06-13 Thread Josef Johansson
Hey, Just examing what happened to an OSD, that was just turned off. Data has been moved away from it, so hesitating to turned it back on. Got the below in the logs, any clues to what the assert talks about? Cheers, Josef -1 os/FileStore.cc: In function 'virtual int FileStore::read(coll_t,