I'm afraid the exact error got lost out of scrollback before I thought to save it. I was running radosgw in debug mode (-d --debug-rgw=20 --debug-ms=20), and it was along the lines of this: 2014-03-19 7faeb7fff700 20 get_obj_state: rctx=0x7fae78002df0 obj=.rgw:productimages state=0x7fae7800a368 s->prefetch_data=0 2014-03-19 1 ... <== osd.3 .. ==== osd_op_reply(... ack = -5 (i/o error)) Not sure how much help that is though.
On 19 March 2014 18:55, Gregory Farnum <g...@inktank.com> wrote: > Exactly what errors did you see, from which log? In general the OSD > does suicide on filesystem errors. > -Greg > Software Engineer #42 @ http://inktank.com | http://ceph.com > > > On Wed, Mar 19, 2014 at 4:06 AM, Mike Bryant <m...@mikebryant.me.uk> > wrote: > > So I've done some more digging, and running the radosgw in debug mode I > > found some messages from osd.3 saying IOError, when it was trying to get > > .rgw:productimages. > > I took that OSD down, and everything started working. > > > > My question now is, why didn't that OSD suicide when it hit an IOError, > > instead of causing the cluster to stop working? > > > > > > On 19 March 2014 10:07, Mike Bryant <m...@mikebryant.me.uk> wrote: > >> > >> Hi, > >> I've just upgraded a test cluster to Emporer, and one of my S3 buckets > >> seems to have broken. > >> > >> s3 access is returning a 500 code (UnknownError). > >> > >> Running bucket stats, it's missing from the list. > >> Trying to do it explicitly: > >> > >> radosgw-admin bucket stats --bucket=productimages > >> 2014-03-19 10:06:17.829397 7ff0b81c7780 0 could not get bucket info for > >> bucket=productimages > >> > >> I can see the header object in the .rgw pool: > >> rados --cluster=cit-external ls --pool .rgw > >> .pools.avail > >> productimages > >> test > >> tests3fs > >> > >> Does anyone have any idea on what might have happened, or how I can get > >> this bucket back? > >> > >> Cheers > >> Mike > > > > > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
