On Wed, Sep 2, 2020 at 12:41 AM Szabo, Istvan (Agoda) <istvan.sz...@agoda.com> wrote: > > Hi, > > > Could someone help me what is the issue with our dployment steps please? > > > Initial RGW Cluster_1 > ===================================================================================ADD_RGW_TO_CLUSTERCreate > Default Realm > - sudo radosgw-admin realm create --rgw-realm=default --defaultCreate Default > Zone Group > - sudo radosgw-admin zonegroup create --rgw-zonegroup=default --master > --defaultCreate Default Zone > - sudo radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=default > --master --defaultUpdate Period > - sudo radosgw-admin period update --rgw-realm=default --commitCreate RGW on > 3 OSD_Node > - sudo ceph orch apply rgw default default --placement=“3 bk-otpsmon-1001 > bk-otpsmon-1002 bk-otpsmon-1003” > ===================================================================================Initial > RGW Cluster_2 > - sudo ceph orch apply rgw default default --placement=“3 hy-otpsmon-2001 > hy-otpsmon-2002 bk-otpsmon-2003" > > > > HOW_TO_CONFIGURE_CEPH_OCTOPUS_MULTISITE > =================================================================================================== > > configure multisite > > radosgw-admin realm create \ > --rgw-realm=agoda \ > --default > > radosgw-admin zonegroup create \ > --rgw-zonegroup=data \ > --endpoints=http://bk-otpsmon-1001:80,http://bk-otpsmon-1002:80,http://bk-otpsmon-1003:80 > \ > --rgw-realm=agoda \ > --master \ > --default > > ***** BUG Run Command-line => Can't send shot command-line ***** > radosgw-admin zone create --rgw-zonegroup=data --rgw-zone=bk --master > --default > --endpoints=http://bk-otpsmon-1001:80,http://bk-otpsmon-1002:80,http://bk-otpsmon-1003:80 > > ***** BUG Can't Delete Pool ***** > ***** ceph config set mon mon_allow_pool_delete true > ***** ceph tell mon.\* injectargs '--mon-allow-pool-delete=true' > ***** (optional) systemctl |grep -i ceph > ***** (optional) systemctl restart > ceph-85b0c358-e2a2-11ea-8864-000c29fa9...@mon.bk-otpsmon-1001.service > > # radosgw-admin zonegroup remove --rgw-zonegroup=default --rgw-zone=default > # radosgw-admin period update --commit > # radosgw-admin zone delete --rgw-zone=default > # radosgw-admin period update --commit > # radosgw-admin zonegroup delete --rgw-zonegroup=default > # radosgw-admin period update --commit > > # ceph osd pool rm default.rgw.meta default.rgw.meta > --yes-i-really-really-mean-it > # ceph osd pool rm default.rgw.control default.rgw.control > --yes-i-really-really-mean-it > # ceph osd pool rm default.rgw.log default.rgw.log > --yes-i-really-really-mean-it > > radosgw-admin user create --uid="ceph-sync" --display-name="ceph-sync" > --system > > "user": "ceph-sync", > "access_key": "VWV6566957QGVDV6ITJM", > "secret_key": "qAEGXslUHBeWv7O6VMCmdo0z2AgMyBZlcKqg38H7" > > radosgw-admin zone modify \ > --rgw-zone=bk \ > --access-key=VWV6566957QGVDV6ITJM \ > --secret=qAEGXslUHBeWv7O6VMCmdo0z2AgMyBZlcKqg38H7 > > radosgw-admin period update --rgw-realm=agoda --commit > > Configure Rados Gateway Client at RGW Nodes > > [client.rgw.bk-otpsmon-1001] > host = bk-otpsmon-1001 > rgw frontends = "civetweb port=80" > rgw_zone=bk > > [client.rgw.bk-otpsmon-1002] > host = bk-otpsmon-1002 > rgw frontends = "civetweb port=80" > rgw_zone=bk > > [client.rgw.bk-otpsmon-1003] > host = bk-otpsmon-1003 > rgw frontends = "civetweb port=80" > rgw_zone=bk > > Restart All RGW Container Node > > rgw_1: systemctl restart > ceph-89c72a6c-eb95-11ea-b88b-000c29147...@rgw.default.default.bk-otpsmon-1001.atdfmv.service > rgw_2: systemctl restart > ceph-89c72a6c-eb95-11ea-b88b-000c29147...@rgw.default.default.bk-otpsmon-1002.wkieqj.service > rgw_3: systemctl restart > ceph-89c72a6c-eb95-11ea-b88b-000c29147...@rgw.default.default.bk-otpsmon-1003.jpyzdq.service > > Enable All RGW Container Node > > rgw_1: systemctl enable > ceph-89c72a6c-eb95-11ea-b88b-000c29147...@rgw.default.default.bk-otpsmon-1001.atdfmv.service > rgw_2: systemctl enable > ceph-89c72a6c-eb95-11ea-b88b-000c29147...@rgw.default.default.bk-otpsmon-1002.wkieqj.service > rgw_3: systemctl enable > ceph-89c72a6c-eb95-11ea-b88b-000c29147...@rgw.default.default.bk-otpsmon-1003.jpyzdq.service > > ===================================== > Secondary Zones > > radosgw-admin realm pull --url={url-to-master-zone-gateway} > --access-key={access-key} --secret={secret} > > radosgw-admin realm pull --url=http://bk-otpsmon-1001:80 > --access-key=CL7NF0DLYL7D2YYVR9HA > --secret=C49mBXDNgHl9fNwibdaQamvffB9QM2RNj5snUq03 > > radosgw-admin realm pull --url=http://bk-otpsmon-1001:80 > --access-key=CL7NF0DLYL7D2YYVR9HA > --secret=C49mBXDNgHl9fNwibdaQamvffB9QM2RNj5snUq03 --rgw-realm=agoda >
I'm not sure where these keys came from. They should be the ones associated with the ceph-sync user created above. > > And here is the errors: > > Error From Cluster_2 After Realm Pull[root@hy-otpsmon-2001 ~]# radosgw-admin > realm pull --url=http://bk-otpsmon-1001:80<http://bk-otpsmon-1001/> > --access-key=CL7NF0DLYL7D2YYVR9HA > --secret=C49mBXDNgHl9fNwibdaQamvffB9QM2RNj5snUq03 --rgw-realm=agoda > request failed: (13) Permission denied > If the realm has been changed on the master zone, the master zone's gateway > may need to be restarted to recognize this user. > [root@hy-otpsmon-2001 ~]# > > Error From Cluster_1 RGW Container LogSep 02 11:20:52 bk-otpsmon-1001 > bash[1246]: debug 2020-09-02T04:20:52.963+0000 7f3b0747f700 1 ====== > starting new request req=0x7f3b455118a0 ===== > Sep 02 11:20:52 bk-otpsmon-1001 bash[1246]: debug > 2020-09-02T04:20:52.967+0000 7f3b0747f700 1 op->ERRORHANDLER: err_no=-2028 > new_err_no=-2028 > Sep 02 11:20:52 bk-otpsmon-1001 bash[1246]: debug > 2020-09-02T04:20:52.967+0000 7f3b0747f700 1 ====== req done > req=0x7f3b455118a0 op status=0 http_status=403 latency=0.002999559s ====== > > All help appreciated. > > > ________________________________ > This message is confidential and is for the sole use of the intended > recipient(s). It may also be privileged or otherwise protected by copyright > or other legal rules. If you have received it by mistake please let us know > by reply email and delete it from your system. It is prohibited to copy this > message or disclose its content to anyone. Any confidentiality or privilege > is not waived or lost by any mistaken delivery or unauthorized disclosure of > the message. All messages sent to and from Agoda may be monitored to ensure > compliance with company policies, to protect the company's interests and to > remove potential malware. Electronic messages may be intercepted, amended, > lost or deleted, or contain viruses. > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
