Hi Robert,
thanks for the update, it's great that the issue is resolved.
Zitat von Robert Kihlberg :
Thanks Josh and Eugen,
I did not manage to trace this object to an S3 object. Instead I read all
files in the suspected S3 bucket and
actually hit a bad one. Since we had a known good mirror I
Thanks Josh and Eugen,
I did not manage to trace this object to an S3 object. Instead I read all
files in the suspected S3 bucket and
actually hit a bad one. Since we had a known good mirror I deleted the
broken S3 object (which succeeded fine)
and uploaded the good one (also succeeded). Data wise
This reminds me of this tracker:
https://tracker.ceph.com/issues/50351
IIRC, the information could be actually lost on the OSDs. I’m
surprised that the number of missing shards is that high though. If
you have the objects mirrored, maybe importing it with
objectstore-tool could be a way fo
We saw this a fair bit in Nautilus, and I also suspected that there
was something up with GC'd and/or deleted objects, but we never
determined the cause. Notably it seemed to happen on PGs ending in
'ff' or 'fff', which was extra suspicious. We haven't seen it since
Pacific.
Josh
On Fri, Oct 4, 2
