Hello Yehuda, The issue seem to be with the user data file for swift subser not getting synced properly. MasterZone:
root@ceph-all:/var/local# ceph osd map .us-1-east-1.users.uid johndoe2 osdmap e796 pool '.us-1-east-1.users.uid' (286) object 'johndoe2' -> pg 286.c384ed51 (286.51) -> up [2] acting [2] root@ceph-all:/var/local# *strings /var/local/osd2/current/286.51_head/johndoe2__head_C384ED51__11e * johndoe2 KYNM9FLJAFNKCB9HN7UG( dwQpa4dWCXXJqGZWcNTBo7SzAFFaRX2LWvMtM3so John Doe_2 joh...@example.com johndoe2:swift( J8tmIvmszn8NEtBOKAHa4zOoJO65C4DoIm6Sl9kt johndoe2 KYNM9FLJAFNKCB9HN7UG KYNM9FLJAFNKCB9HN7UG( dwQpa4dWCXXJqGZWcNTBo7SzAFFaRX2LWvMtM3so swift swift *johndoe2:swift* johndoe2:swift( J8tmIvmszn8NEtBOKAHa4zOoJO65C4DoIm6Sl9kt swift Slave Zone After Replication: root@ceph-all-1:/var/chef/cache# ceph osd map .us-1-west-1.users.uid johndoe2 osdmap e202 pool '.us-1-west-1.users.uid' (63) object 'johndoe2' -> pg 63.c384ed51 (63.51) -> up [2] acting [2] root@ceph-all-1:/var/chef/cache# *strings /var/local/osd2/current/63.51_head/johndoe2__head_C384ED51__3f* johndoe2 KYNM9FLJAFNKCB9HN7UG( dwQpa4dWCXXJqGZWcNTBo7SzAFFaRX2LWvMtM3so John Doe_2 joh...@example.com johndoe2:swift( J8tmIvmszn8NEtBOKAHa4zOoJO65C4DoIm6Sl9kt johndoe2 KYNM9FLJAFNKCB9HN7UG KYNM9FLJAFNKCB9HN7UG( dwQpa4dWCXXJqGZWcNTBo7SzAFFaRX2LWvMtM3so swift swift *swift* johndoe2:swift( J8tmIvmszn8NEtBOKAHa4zOoJO65C4DoIm6Sl9kt swift Look @ the diff in bold and even Checksum and Filesizes are different MasterZone: root@ceph-all:/var/local# ls -l /var/local/osd2/current/286.51_head/johndoe2__head_C384ED51__11e -rw-r--r-- 1 root root *469* Jan 6 19:13 /var/local/osd2/current/286.51_head/johndoe2__head_C384ED51__11e SlaveZone: root@ceph-all-1:/var/chef/cache# ls -l /var/local/osd2/current/63.51_head/johndoe2__head_C384ED51__3f -rw-r--r-- 1 root root *460* Jan 6 19:22 /var/local/osd2/current/63.51_head/johndoe2__head_C384ED51__3f -Hemant On Wed, Jan 7, 2015 at 9:27 AM, Mark Kirkwood <mark.kirkw...@catalyst.net.nz > wrote: > On 07/01/15 16:22, Mark Kirkwood wrote: > >> >> >> FWIW I can reproduce this too (ceph 0.90-663-ge1384af). The *user* >> replicates ok (complete with its swift keys and secret). I can >> authenticate to both zones ok using S3 api (boto version 2.29), but only >> to the master using swift (swift client versions 2.3.1 and 2.0.3). In >> the case of the slave zone I'm seeing the same error stack as the above. >> >> I'm running Ubuntu 14.10 for ceph and rgw with Apache (version 2.4.10) >> the standard repos. I'll try replacing the fastcgi module to see if that >> is a factor. >> >> > Does not appear to be - downgraded apache to 2.4.7 + fastcgi from ceph > repo and seeing the same sort of thing on the slave zone: > > > 2015-01-07 16:56:13.889445 7febba77c700 1 ====== starting new request > req=0x7fec34075ba0 ===== > 2015-01-07 16:56:13.889456 7febba77c700 2 req 2454:0.000010::GET > /auth::initializing > 2015-01-07 16:56:13.889461 7febba77c700 10 host=192.168.122.21 > rgw_dns_name=ceph1 > 2015-01-07 16:56:13.889475 7febba77c700 2 req > 2454:0.000030:swift-auth:GET /auth::getting op > 2015-01-07 16:56:13.889480 7febba77c700 2 req > 2454:0.000034:swift-auth:GET /auth:swift_auth_get:authorizing > 2015-01-07 16:56:13.889481 7febba77c700 2 req > 2454:0.000036:swift-auth:GET /auth:swift_auth_get:reading permissions > 2015-01-07 16:56:13.889482 7febba77c700 2 req > 2454:0.000037:swift-auth:GET /auth:swift_auth_get:init op > 2015-01-07 16:56:13.889484 7febba77c700 2 req > 2454:0.000038:swift-auth:GET /auth:swift_auth_get:verifying op mask > 2015-01-07 16:56:13.889485 7febba77c700 20 required_mask= 0 user.op_mask=7 > 2015-01-07 16:56:13.889486 7febba77c700 2 req > 2454:0.000041:swift-auth:GET /auth:swift_auth_get:verifying op permissions > 2015-01-07 16:56:13.889487 7febba77c700 2 req > 2454:0.000042:swift-auth:GET /auth:swift_auth_get:verifying op params > 2015-01-07 16:56:13.889488 7febba77c700 2 req > 2454:0.000043:swift-auth:GET /auth:swift_auth_get:executing > 2015-01-07 16:56:13.889516 7febba77c700 2 req > 2454:0.000070:swift-auth:GET /auth:swift_auth_get:http status=403 > 2015-01-07 16:56:13.889518 7febba77c700 1 ====== req done > req=0x7fec34075ba0 http_status=403 ====== > 2015-01-07 16:56:13.889521 7febba77c700 20 process_request() returned -1 > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
