I fixed this inconsistent error. It seems ceph didn't delete the mismatch
object that depends the deleted snapshot. This caused error "unexpected
clone" that resulted of message inconsistent.
Log:
2017-12-12 20:14:06.651942 7fc7eff7e700 -1 log_channel(cluster) log [ERR] :
deep-scrub 4.1b42
4:42db3cde:::rbd_data.aa5af9238e1f29.000000000000470a:2ccac is an
unexpected clone
Steps to fix:
1. Move the mismatch object on both of osd.159 and osd.179 to other location
2. ceph pg deep-scrub 4.1b42
On Tue, Dec 12, 2017 at 9:34 PM, Thanh Tran <cep...@gmail.com> wrote:
> Hi,
>
> My ceph cluster has a inconsistent pg. I tried to deep scrub pg and repair
> pg but not fix problems.
> I found that the object that made pg inconsistent depends on snapshot
> (snap id is 2ccac = 183468) of a image, I deleted this snapshot, then query
> inconsistent pg and it showed empty, but my ceph cluster still show error
> inconsistent pg. It's strange that the mismatch object depends on the
> deleted snapshot still exist on disk.
>
> After deleting the snapshot has the mismatch object, I re-tried to deep
> scrub and repair pg but it doesn't help to fix.
>
> Information of commands as below. I don't know how to clear this status of
> inconsistent. Please help me to fix this bug.
>
> *ceph --cluster QTC01 health detail*
> HEALTH_ERR 1 pgs inconsistent; 1 scrub errors
> pg 4.1b42 is active+clean+inconsistent, acting [159,179]
> 1 scrub errors
>
> *rados list-inconsistent-obj 4.1b42 --format=json-pretty --cluster QTC01*
> {
> "epoch": 494830,
> "inconsistents": []
> }
>
> *ll /var/lib/ceph/osd/QTC01-159/current/4.1b42_head/DIR_2/DIR_4/DIR_B/ |
> grep aa5af9238e1f29*
> -rw-r--r-- 1 ceph ceph 4194304 Dec 12 15:29 rbd\udata.aa5af9238e1f29.
> 000000000000470a__2ccac_7B3CDB42__4
> -rw-r--r-- 1 ceph ceph 1048576 Dec 12 15:29 rbd\udata.aa5af9238e1f29.
> 000000000000470a__head_7B3CDB42__4
>
> *ll /var/lib/ceph/osd/QTC01-179/current/4.1b42_head/DIR_2/DIR_4/DIR_B/ |
> grep aa5af9238e1f29*
> -rw-r--r-- 1 ceph ceph 4194304 Dec 12 15:17 rbd\udata.aa5af9238e1f29.
> 000000000000470a__2ccac_7B3CDB42__4
> -rw-r--r-- 1 ceph ceph 1048576 Dec 12 15:17 rbd\udata.aa5af9238e1f29.
> 000000000000470a__head_7B3CDB42__4
>
> Best regards,
> Thanh Tran
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
