Hello Geeks Need your help and advice in this problem.
- VS - On Tue, Apr 28, 2015 at 12:48 AM, Vickey Singh <vickey.singh22...@gmail.com> wrote: > Hello Alfredo / Craig > > First of all Thank You So much for replying and giving your precious time > to this problem. > > @Alfredo : I tried version radosgw-agent version 1.2.2 and the case has > progressed a lot. ( below are some the logs ) > > > I am now getting > > *2015-04-28 00:35:14,781 5132 [radosgw_agent][INFO ] > http://us-east-1.crosslogic.com:7480 <http://us-east-1.crosslogic.com:7480> > endpoint does not support versioning* > > *2015-04-28 00:35:14,781 5132 [radosgw_agent][WARNIN] encountered issues > reaching to endpoint http://us-east-1.crosslogic.com:7480 > <http://us-east-1.crosslogic.com:7480>* > > *2015-04-28 00:35:14,782 5132 [radosgw_agent][WARNIN] HTTP Error 403: > Forbidden* > > I am using CIVETWEB , any further help in this would be really helpful. > > > [root@us-east-1 ceph]# > > [root@us-east-1 ceph]# radosgw-agent -c cluster-data-sync.conf > > 2015-04-28 00:35:14,750 5132 [radosgw_agent][INFO ] __ __ > __ ___ ___ > > 2015-04-28 00:35:14,750 5132 [radosgw_agent][INFO ] /__` \ / |\ | / ` > /\ / _` |__ |\ | | > > 2015-04-28 00:35:14,751 5132 [radosgw_agent][INFO ] .__/ | | \| \__, > /~~\ \__> |___ | \| | > > 2015-04-28 00:35:14,751 5132 [radosgw_agent][INFO ] > v1.2.2 > > 2015-04-28 00:35:14,751 5132 [radosgw_agent][INFO ] agent options: > > 2015-04-28 00:35:14,752 5132 [radosgw_agent][INFO ] args: > > 2015-04-28 00:35:14,753 5132 [radosgw_agent][INFO ] conf > : None > > 2015-04-28 00:35:14,753 5132 [radosgw_agent][INFO ] dest_access_key > : **************** > > 2015-04-28 00:35:14,753 5132 [radosgw_agent][INFO ] dest_secret_key > : **************** > > 2015-04-28 00:35:14,753 5132 [radosgw_agent][INFO ] destination > : http://us-west-1.crosslogic.com:7480 > > 2015-04-28 00:35:14,753 5132 [radosgw_agent][INFO ] > incremental_sync_delay : 30 > > 2015-04-28 00:35:14,754 5132 [radosgw_agent][INFO ] lock_timeout > : 60 > > 2015-04-28 00:35:14,754 5132 [radosgw_agent][INFO ] log_file > : /var/log/radosgw/radosgw-sync-us-east-west.log > > 2015-04-28 00:35:14,756 5132 [radosgw_agent][INFO ] log_lock_time > : 20 > > 2015-04-28 00:35:14,756 5132 [radosgw_agent][INFO ] max_entries > : 1000 > > 2015-04-28 00:35:14,757 5132 [radosgw_agent][INFO ] metadata_only > : False > > 2015-04-28 00:35:14,757 5132 [radosgw_agent][INFO ] num_workers > : 1 > > 2015-04-28 00:35:14,758 5132 [radosgw_agent][INFO ] > object_sync_timeout : 216000 > > 2015-04-28 00:35:14,758 5132 [radosgw_agent][INFO ] > prepare_error_delay : 10 > > 2015-04-28 00:35:14,758 5132 [radosgw_agent][INFO ] quiet > : False > > 2015-04-28 00:35:14,758 5132 [radosgw_agent][INFO ] > rgw_data_log_window : 30 > > 2015-04-28 00:35:14,759 5132 [radosgw_agent][INFO ] source > : None > > 2015-04-28 00:35:14,759 5132 [radosgw_agent][INFO ] src_access_key > : **************** > > 2015-04-28 00:35:14,759 5132 [radosgw_agent][INFO ] src_secret_key > : **************** > > 2015-04-28 00:35:14,759 5132 [radosgw_agent][INFO ] src_zone > : None > > 2015-04-28 00:35:14,759 5132 [radosgw_agent][INFO ] sync_scope > : incremental > > 2015-04-28 00:35:14,760 5132 [radosgw_agent][INFO ] test_server_host > : None > > 2015-04-28 00:35:14,760 5132 [radosgw_agent][INFO ] test_server_port > : 8080 > > 2015-04-28 00:35:14,761 5132 [radosgw_agent][INFO ] verbose > : False > > 2015-04-28 00:35:14,761 5132 [radosgw_agent][INFO ] versioned > : False > > 2015-04-28 00:35:14,761 5132 [radosgw_agent.client][INFO ] creating > connection to endpoint: http://us-west-1.crosslogic.com:7480 > > region map is: {u'us': [u'us-west', u'us-east']} > > *2015-04-28 00:35:14,781 5132 [radosgw_agent][INFO ] > http://us-east-1.crosslogic.com:7480 <http://us-east-1.crosslogic.com:7480> > endpoint does not support versioning* > > *2015-04-28 00:35:14,781 5132 [radosgw_agent][WARNIN] encountered issues > reaching to endpoint http://us-east-1.crosslogic.com:7480 > <http://us-east-1.crosslogic.com:7480>* > > *2015-04-28 00:35:14,782 5132 [radosgw_agent][WARNIN] HTTP Error 403: > Forbidden* > > 2015-04-28 00:35:14,782 5132 [radosgw_agent.client][INFO ] creating > connection to endpoint: http://us-east-1.crosslogic.com:7480 > > 2015-04-28 00:35:14,784 5132 [radosgw_agent.client][INFO ] creating > connection to endpoint: http://us-west-1.crosslogic.com:7480 > > 2015-04-28 00:35:14,785 5132 [radosgw_agent.client][INFO ] creating > connection to endpoint: http://us-east-1.crosslogic.com:7480 > > 2015-04-28 00:35:14,787 5132 [radosgw_agent.client][INFO ] creating > connection to endpoint: http://us-west-1.crosslogic.com:7480 > > *2015-04-28 00:35:14,807 5132 [radosgw_agent.sync][ERROR ] finding number > of shards failed* > > 2015-04-28 00:35:14,807 5132 [radosgw_agent.sync][WARNIN] error preparing > for sync, will retry. Traceback: > > Traceback (most recent call last): > > File "/usr/lib/python2.7/site-packages/radosgw_agent/sync.py", line 30, > in prepare_sync > > syncer.prepare() > > File "/usr/lib/python2.7/site-packages/radosgw_agent/sync.py", line 208, > in prepare > > self.init_num_shards() > > File "/usr/lib/python2.7/site-packages/radosgw_agent/sync.py", line 77, > in init_num_shards > > self.num_shards = client.num_log_shards(self.src_conn, self.type) > > File "/usr/lib/python2.7/site-packages/radosgw_agent/client.py", line > 403, in num_log_shards > > out = request(connection, 'get', 'admin/log', dict(type=shard_type)) > > File "/usr/lib/python2.7/site-packages/radosgw_agent/client.py", line > 147, in request > > check_result_status(result) > > File "/usr/lib/python2.7/site-packages/radosgw_agent/client.py", line > 89, in check_result_status > > exc.HttpError)(result.status, result.reason) > > *HttpError: Http error code 403 content Forbidden* > > *2015-04-28 00:35:24,826 5132 [radosgw_agent.sync][ERROR ] finding number > of shards failed* > > 2015-04-28 00:35:24,826 5132 [radosgw_agent.sync][WARNIN] error preparing > for sync, will retry. Traceback: > > Traceback (most recent call last): > > File "/usr/lib/python2.7/site-packages/radosgw_agent/sync.py", line 30, > in prepare_sync > > syncer.prepare() > > File "/usr/lib/python2.7/site-packages/radosgw_agent/sync.py", line 208, > in prepare > > self.init_num_shards() > > File "/usr/lib/python2.7/site-packages/radosgw_agent/sync.py", line 77, > in init_num_shards > > self.num_shards = client.num_log_shards(self.src_conn, self.type) > > File "/usr/lib/python2.7/site-packages/radosgw_agent/client.py", line > 403, in num_log_shards > > out = request(connection, 'get', 'admin/log', dict(type=shard_type)) > > File "/usr/lib/python2.7/site-packages/radosgw_agent/client.py", line > 147, in request > > check_result_status(result) > > File "/usr/lib/python2.7/site-packages/radosgw_agent/client.py", line > 89, in check_result_status > > exc.HttpError)(result.status, result.reason) > > HttpError: Http error code 403 content Forbidden > > ^C2015-04-28 00:35:25,609 5132 [radosgw_agent][ERROR ] KeyboardInterrupt > > > [root@us-east-1 ceph]# > > > > [root@us-east-1 ceph]# curl http://us-east-1.crosslogic.com:7480 > > <?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns=" > http://s3.amazonaws.com/doc/2006-03-01/ > "><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult> > > [root@us-east-1 ceph]# > > [root@us-east-1 ceph]# > > > [root@us-east-1 ceph]# curl http://us-west-1.crosslogic.com:7480 > > <?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns=" > http://s3.amazonaws.com/doc/2006-03-01/ > "><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult> > > [root@us-east-1 ceph]# > > [root@us-east-1 ceph]# > > > > [root@us-east-1 ceph]# radosgw-admin region-map get --name > client.radosgw.us-east-1 > > { "regions": [ > > { "key": "us", > > "val": { "name": "us", > > "api_name": "us", > > "is_master": "true", > > "endpoints": [ > > "http:\/\/us-east-1.crosslogic.com:7480\/"], > > "master_zone": "us-east", > > "zones": [ > > { "name": "us-east", > > "endpoints": [ > > "http:\/\/us-east-1.crosslogic.com:7480\/"], > > "log_meta": "true", > > "log_data": "true"}, > > { "name": "us-west", > > "endpoints": [ > > "http:\/\/us-west-1.crosslogic.com:7480\/"], > > "log_meta": "true", > > "log_data": "true"}], > > "placement_targets": [ > > { "name": "default-placement", > > "tags": []}], > > "default_placement": "default-placement"}}], > > "master_region": "us", > > "bucket_quota": { "enabled": false, > > "max_size_kb": -1, > > "max_objects": -1}, > > "user_quota": { "enabled": false, > > "max_size_kb": -1, > > "max_objects": -1}}[root@us-east-1 ceph]# > > [root@us-east-1 ceph]# > > > > [root@us-east-1 ceph]# radosgw-admin region-map get --name > client.radosgw.us-west-1 > > { "regions": [ > > { "key": "us", > > "val": { "name": "us", > > "api_name": "us", > > "is_master": "true", > > "endpoints": [ > > "http:\/\/us-east-1.crosslogic.com:7480\/"], > > "master_zone": "us-east", > > "zones": [ > > { "name": "us-east", > > "endpoints": [ > > "http:\/\/us-east-1.crosslogic.com:7480\/"], > > "log_meta": "true", > > "log_data": "true"}, > > { "name": "us-west", > > "endpoints": [ > > "http:\/\/us-west-1.crosslogic.com:7480\/"], > > "log_meta": "true", > > "log_data": "true"}], > > "placement_targets": [ > > { "name": "default-placement", > > "tags": []}], > > "default_placement": "default-placement"}}], > > "master_region": "us", > > "bucket_quota": { "enabled": false, > > "max_size_kb": -1, > > "max_objects": -1}, > > "user_quota": { "enabled": false, > > "max_size_kb": -1, > > "max_objects": -1}}[root@us-east-1 ceph]# > > [root@us-east-1 ceph]# > > - VS - > > > > On Mon, Apr 27, 2015 at 11:59 PM, Alfredo Deza <ad...@redhat.com> wrote: > >> Hi Vickey (and all) >> >> It looks like this issue was introduced as part of the 1.2.1 release. >> >> I just finished getting 1.2.2 out (try upgrading please). You should no >> longer see that >> error. >> >> Hope that helps! >> >> -Alfredo >> >> ----- Original Message ----- >> From: "Craig Lewis" <cle...@centraldesktop.com> >> To: "Vickey Singh" <vickey.singh22...@gmail.com> >> Cc: ceph-users@lists.ceph.com >> Sent: Monday, April 27, 2015 4:23:52 PM >> Subject: Re: [ceph-users] Ceph Radosgw multi zone data replication failure >> >> > [root@us-east-1 ceph] # ceph -s --name client.radosgw.us-east-1 >> >> >> > [root@us-east-1 ceph]# ceph -s --name client.radosgw.us-west-1 >> >> Are you trying to setup two zones on one cluster? That's possible, but >> you'll also want to spend some time on your CRUSH map making sure that the >> two zones are as independent as possible (no shared disks, etc). >> >> Are you using Civetweb or Apache + FastCGI? >> >> Can you include the output (from both clusters): >> radosgw-admin --name=client.radosgw.us-east-1 region get >> radosgw-admin --name=client.radosgw.us-east-1 zone get >> >> Double check that both system users exist in both clusters, with the same >> secret. >> >> >> >> >> On Sun, Apr 26, 2015 at 8:01 AM, Vickey Singh < >> vickey.singh22...@gmail.com > wrote: >> >> >> >> >> >> Hello Geeks >> >> >> >> >> I am trying to setup Ceph Radosgw multi site data replication using >> official documentation >> http://ceph.com/docs/master/radosgw/federated-config/#multi-site-data-replication >> >> >> >> >> Everything seems to work except radosgw-agent sync , Request you to >> please check the below outputs and help me in any possible way. >> >> >> >> >> Environment : >> >> >> >> >> CentOS 7.0.1406 >> >> Ceph Versino 0.87.1 >> >> Rados Gateway configured using Civetweb >> >> >> >> >> >> >> >> Radosgw zone list : Works nicely >> >> >> >> >> >> [root@us-east-1 ceph]# radosgw-admin zone list --name >> client.radosgw.us-east-1 >> >> { "zones": [ >> >> "us-west", >> >> "us-east"]} >> >> [root@us-east-1 ceph]# >> >> >> >> >> Curl request to master zone : Works nicely >> >> >> >> >> >> [root@us-east-1 ceph]# curl http://us-east-1.crosslogic.com:7480 >> >> <?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns=" >> http://s3.amazonaws.com/doc/2006-03-01/ >> "><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult> >> >> [root@us-east-1 ceph]# >> >> >> >> >> Curl request to secondary zone : Works nicely >> >> >> >> >> [root@us-east-1 ceph]# curl http://us-west-1.crosslogic.com:7480 >> >> <?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns=" >> http://s3.amazonaws.com/doc/2006-03-01/ >> "><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult> >> >> [root@us-east-1 ceph]# >> >> >> >> >> Rados Gateway agent configuration file : Seems correct, no TYPO errors >> >> >> >> >> >> [root@us-east-1 ceph] # cat cluster-data-sync.conf >> >> src_access_key: M7QAKDH8CYGTK86CG93U >> >> src_secret_key: 0xQR6PINk23W\/GYrWJ14aF+1stG56M6xMkqkdloO >> >> destination: http://us-west-1.crosslogic.com:7480 >> >> dest_access_key: ZQ32ES1WAWPG05YMZ7T7 >> >> dest_secret_key: INvk8AkrZRsejLEL34yRpMLmOqydt8ncOXy4RHCM >> >> log_file: /var/log/radosgw/radosgw-sync-us-east-west.log >> >> [root@us-east-1 ceph]# >> >> >> >> >> Rados Gateway agent SYNC : Fails , however it can fetch region map so i >> think src and dest KEYS are correct. But don't know why it fails on >> AttributeError >> >> >> >> >> >> [root@us-east-1 ceph]# radosgw-agent -c cluster-data-sync.conf >> >> >> region map is: {u'us': [u'us-west', u'us-east']} >> >> Traceback (most recent call last): >> >> File "/usr/bin/radosgw-agent", line 21, in <module> >> >> sys.exit(main()) >> >> File "/usr/lib/python2.7/site-packages/radosgw_agent/cli.py", line 275, >> in main >> >> except client.ClientException as e: >> >> AttributeError: 'module' object has no attribute 'ClientException' >> >> [root@us-east-1 ceph]# >> >> >> >> >> Can query to Ceph cluster using us-east-1 ID >> >> >> >> >> [root@us-east-1 ceph] # ceph -s --name client.radosgw.us-east-1 >> >> cluster 9609b429-eee2-4e23-af31-28a24fcf5cbc >> >> health HEALTH_OK >> >> monmap e3: 3 mons at {ceph-node1= >> 192.168.1.101:6789/0,ceph-node2=192.168.1.102:6789/0,ceph-node3=192.168.1.103:6789/0 >> }, election epoch 448, quorum 0,1,2 ceph-node1,ceph-node2,ceph-node3 >> >> osdmap e1063: 9 osds: 9 up, 9 in >> >> pgmap v8473: 1500 pgs, 43 pools, 374 MB data, 2852 objects >> >> 1193 MB used, 133 GB / 134 GB avail >> >> 1500 active+clean >> >> [root@us-east-1 ceph]# >> >> >> >> >> Can query to Ceph cluster using us-west-1 ID >> >> >> >> >> >> [root@us-east-1 ceph]# ceph -s --name client.radosgw.us-west-1 >> >> cluster 9609b429-eee2-4e23-af31-28a24fcf5cbc >> >> health HEALTH_OK >> >> monmap e3: 3 mons at {ceph-node1= >> 192.168.1.101:6789/0,ceph-node2=192.168.1.102:6789/0,ceph-node3=192.168.1.103:6789/0 >> }, election epoch 448, quorum 0,1,2 ceph-node1,ceph-node2,ceph-node3 >> >> osdmap e1063: 9 osds: 9 up, 9 in >> >> pgmap v8473: 1500 pgs, 43 pools, 374 MB data, 2852 objects >> >> 1193 MB used, 133 GB / 134 GB avail >> >> 1500 active+clean >> >> [root@us-east-1 ceph]# >> >> >> >> >> Hope these packages are correct >> >> >> >> >> >> [root@us-east-1 ceph]# rpm -qa | egrep -i "ceph|radosgw" >> >> libcephfs1-0.87.1-0.el7.centos.x86_64 >> >> ceph-common-0.87.1-0.el7.centos.x86_64 >> >> python-ceph-0.87.1-0.el7.centos.x86_64 >> >> ceph-radosgw-0.87.1-0.el7.centos.x86_64 >> >> ceph-release-1-0.el7.noarch >> >> ceph-0.87.1-0.el7.centos.x86_64 >> >> radosgw-agent-1.2.1-0.el7.centos.noarch >> >> [root@us-east-1 ceph]# >> >> >> >> >> >> >> >> Regards >> >> VS >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
