Hi Amit,
Still same. Please see the below output. Anything else I can try? Please update
[cephuser@zabbix-client ~]$ radosgw-admin period update --commit 2>/dev/null
Sending period to new master zone 71931e0e-1be6-449f-af34-edb4166c4e4a
[cephuser@zabbix-client ~]$ sudo systemctl start ceph-radosgw@rgw.`hostname -s`
[cephuser@zabbix-client ~]$ sudo systemctl restart ceph-radosgw@rgw.`hostname
-s`
[cephuser@zabbix-client ~]$ sudo systemctl status ceph-radosgw@rgw.`hostname -s`
● ceph-radosgw@rgw.zabbix-client.service - Ceph rados gateway
Loaded: loaded (/usr/lib/systemd/system/ceph-radosgw@.service; enabled;
vendor preset: disabled)
Active: active (running) since Wed 2019-01-30 22:38:15 IST; 7s ago
Main PID: 8234 (radosgw)
CGroup:
/system.slice/system-ceph\x2dradosgw.slice/ceph-radosgw@rgw.zabbix-client.service
└─8234 /usr/bin/radosgw -f --cluster ceph --name
client.rgw.zabbix-client --setuser ceph --setgroup ceph
Jan 30 22:38:15 zabbix-client systemd[1]: Started Ceph rados gateway.
[cephuser@zabbix-client ~]$ radosgw-admin sync status --source-zone noida1
2>/dev/null
realm 1102c891-d81c-480e-9487-c9f874287d13 (georep)
zonegroup 74ad391b-fbca-4c05-b9e7-c90fd4851223 (noida)
zone 45c690a8-f39c-4b1d-9faf-e0e991ceaaac (san-jose)
metadata sync failed to read sync status: (2) No such file or directory
data sync source: 71931e0e-1be6-449f-af34-edb4166c4e4a (noida1)
failed to retrieve sync info: (5) Input/output error
[cephuser@zabbix-client ~]$
/Krishna
From: Amit Ghadge <amitg....@gmail.com>
Sent: Wednesday, January 30, 2019 9:17 PM
To: Krishna Verma <kve...@cadence.com>
Cc: Casey Bodley <cbod...@redhat.com>; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] Multisite Ceph setup sync issue
EXTERNAL MAIL
Have you commit your changes on slave gateway?
First, run commit command on slave gateway and then try.
-AmitG
On Wed, 30 Jan 2019, 21:06 Krishna Verma,
<kve...@cadence.com<mailto:kve...@cadence.com>> wrote:
Hi Casey,
Thanks for your reply, however I tried with "--source-zone" option with sync
command but getting below error:
Sync status From slave gateway to master zone "noida1"
[cephuser@zabbix-client ~]$ radosgw-admin sync status --source-zone noida1
2>/dev/null
realm 1102c891-d81c-480e-9487-c9f874287d13 (georep)
zonegroup 74ad391b-fbca-4c05-b9e7-c90fd4851223 (noida)
zone 45c690a8-f39c-4b1d-9faf-e0e991ceaaac (san-jose)
metadata sync failed to read sync status: (2) No such file or directory
data sync source: 71931e0e-1be6-449f-af34-edb4166c4e4a (noida1)
failed to retrieve sync info: (5) Input/output error
[cephuser@zabbix-client ~]$
Sync status From Master Gateway to slave zone " san-jose":
[cephuser@zabbix-server ~]$ radosgw-admin sync status --source-zone san-jose
2>/dev/null
realm 1102c891-d81c-480e-9487-c9f874287d13 (georep)
zonegroup 74ad391b-fbca-4c05-b9e7-c90fd4851223 (noida)
zone 71931e0e-1be6-449f-af34-edb4166c4e4a (noida1)
metadata sync no sync (zone is master)
[cephuser@zabbix-server ~]$
Zone detail from master gateway :
[cephuser@zabbix-server ~]$ radosgw-admin zonegroup get 2>/dev/null
{
"id": "74ad391b-fbca-4c05-b9e7-c90fd4851223",
"name": "noida",
"api_name": "noida",
"is_master": "true",
"endpoints": [
"http:\/\/zabbix-server:7480"
],
"hostnames": [],
"hostnames_s3website": [],
"master_zone": "71931e0e-1be6-449f-af34-edb4166c4e4a",
"zones": [
{
"id": "71931e0e-1be6-449f-af34-edb4166c4e4a",
"name": "noida1",
"endpoints": [
"http:\/\/vlno-ceph01:7480"
],
"log_meta": "false",
"log_data": "false",
"bucket_index_max_shards": 0,
"read_only": "false"
}
],
"placement_targets": [
{
"name": "default-placement",
"tags": []
}
],
"default_placement": "default-placement",
"realm_id": "1102c891-d81c-480e-9487-c9f874287d13"
}
[cephuser@zabbix-server ~]$
Zone detail from slave gateway:
[cephuser@zabbix-client ~]$ radosgw-admin zonegroup get 2>/dev/null
{
"id": "74ad391b-fbca-4c05-b9e7-c90fd4851223",
"name": "noida",
"api_name": "noida",
"is_master": "true",
"endpoints": [
"http:\/\/zabbix-server:7480"
],
"hostnames": [],
"hostnames_s3website": [],
"master_zone": "71931e0e-1be6-449f-af34-edb4166c4e4a",
"zones": [
{
"id": "45c690a8-f39c-4b1d-9faf-e0e991ceaaac",
"name": "san-jose",
"endpoints": [
"http:\/\/zabbix-client:7480"
],
"log_meta": "false",
"log_data": "true",
"bucket_index_max_shards": 0,
"read_only": "false"
},
{
"id": "71931e0e-1be6-449f-af34-edb4166c4e4a",
"name": "noida1",
"endpoints": [
"http:\/\/vlno-ceph01:7480"
],
"log_meta": "false",
"log_data": "true",
"bucket_index_max_shards": 0,
"read_only": "false"
}
],
"placement_targets": [
{
"name": "default-placement",
"tags": []
}
],
"default_placement": "default-placement",
"realm_id": "1102c891-d81c-480e-9487-c9f874287d13"
}
[cephuser@zabbix-client ~]
I need your expert advice.
/Krishna
-----Original Message-----
From: Casey Bodley <cbod...@redhat.com<mailto:cbod...@redhat.com>>
Sent: Wednesday, January 30, 2019 1:54 AM
To: Krishna Verma <kve...@cadence.com<mailto:kve...@cadence.com>>
Cc: ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
Subject: Re: [ceph-users] Multisite Ceph setup sync issue
EXTERNAL MAIL
On Tue, Jan 29, 2019 at 12:24 PM Krishna Verma
<kve...@cadence.com<mailto:kve...@cadence.com>> wrote:
>
> Hi Ceph Users,
>
>
>
> I need your to fix sync issue in multisite setup.
>
>
>
> I have 2 cluster in different datacenter that we want to use for
> bidirectional data replication. By followed the documentation
> https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.ceph.com_docs_master_radosgw_multisite_&d=DwIFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=6MZxZ1DTyLWzj88W8kB9g8C3vLhZvcRI3-Xv_HdQ-Hg&s=Uw-eyENNATG6meKsTgmwdwMLYUD13mmDkpr9Eo2dqZo&e=
> I have setup the gateway on each site but when I am checking the sync status
> its getting failed as below:
>
>
>
> Admin node at master :
>
> [cephuser@vlno-ceph01 cluster]$ radosgw-admin data sync status
>
> ERROR: source zone not specified
>
> [cephuser@vlno-ceph01 cluster]$ radosgw-admin realm list
>
> {
>
> "default_info": "1102c891-d81c-480e-9487-c9f874287d13",
>
> "realms": [
>
> "georep",
>
> "geodata"
>
> ]
>
> }
>
>
>
> [cephuser@vlno-ceph01 cluster]$ radosgw-admin zonegroup list
>
> read_default_id : 0
>
> {
>
> "default_info": "74ad391b-fbca-4c05-b9e7-c90fd4851223",
>
> "zonegroups": [
>
> "noida"
>
> ]
>
> }
>
>
>
> [cephuser@vlno-ceph01 cluster]$ radosgw-admin zone list
>
> {
>
> "default_info": "71931e0e-1be6-449f-af34-edb4166c4e4a",
>
> "zones": [
>
> "noida1"
>
> ]
>
> }
>
>
>
> [cephuser@vlno-ceph01 cluster]$
>
>
>
> [cephuser@vlno-ceph01 cluster]$ cat ceph.conf
>
> [global]
>
> fsid = d52e50a4-ed2e-44cc-aa08-9309bc539a55
>
> mon_initial_members = vlno-ceph01
>
> mon_host = 172.23.16.67
>
> auth_cluster_required = cephx
>
> auth_service_required = cephx
>
> auth_client_required = cephx
>
> # Your network address
>
> public network =
> 172.23.16.0/24<https://urldefense.proofpoint.com/v2/url?u=http-3A__172.23.16.0_24&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=h3q181aKjOAdd1PhqHUiPIOe8moxPTxBm5SaOQp8nyQ&s=waKeDUdsZwy6I6S35PbTcm9QVeM1auZqxWSPbxGGhaE&e=>
>
> osd pool default size = 2
>
> rgw_override_bucket_index_max_shards = 100
>
> debug ms = 1
>
> debug rgw = 20
>
> [cephuser@vlno-ceph01 cluster]$
>
>
>
> On Master Gateway :
>
>
>
> [cephuser@zabbix-server ~]$ cat /etc/ceph/ceph.conf
>
> [global]
>
> fsid = d52e50a4-ed2e-44cc-aa08-9309bc539a55
>
> mon_initial_members = vlno-ceph01
>
> mon_host = 172.23.16.67
>
> auth_cluster_required = cephx
>
> auth_service_required = cephx
>
> auth_client_required = cephx
>
> # Your network address
>
> public network =
> 172.23.16.0/24<https://urldefense.proofpoint.com/v2/url?u=http-3A__172.23.16.0_24&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=h3q181aKjOAdd1PhqHUiPIOe8moxPTxBm5SaOQp8nyQ&s=waKeDUdsZwy6I6S35PbTcm9QVeM1auZqxWSPbxGGhaE&e=>
>
> osd pool default size = 2
>
> rgw_override_bucket_index_max_shards = 100
>
> debug ms = 1
>
> debug rgw = 20
>
> [client.rgw.zabbix-server]
>
> host = zabbix-server
>
> rgw frontends = "civetweb port=7480"
>
> rgw_zone=noida1
>
> [cephuser@zabbix-server ~]$
>
>
>
>
>
> On Secondary site admin node.
>
>
>
> [cephuser@vlsj-kverma1 cluster]$ radosgw-admin realm list
>
> {
>
> "default_info": "1102c891-d81c-480e-9487-c9f874287d13",
>
> "realms": [
>
> "georep"
>
> ]
>
> }
>
>
>
> [cephuser@vlsj-kverma1 cluster]$ radosgw-admin zonegroup list
>
> read_default_id : 0
>
> {
>
> "default_info": "74ad391b-fbca-4c05-b9e7-c90fd4851223",
>
> "zonegroups": [
>
> "noida",
>
> "default"
>
> ]
>
> }
>
>
>
> [cephuser@vlsj-kverma1 cluster]$ radosgw-admin zone list
>
> {
>
> "default_info": "45c690a8-f39c-4b1d-9faf-e0e991ceaaac",
>
> "zones": [
>
> "san-jose"
>
> ]
>
> }
>
>
>
> [cephuser@vlsj-kverma1 cluster]$
>
>
>
>
>
> [cephuser@vlsj-kverma1 cluster]$ cat ceph.conf
>
> [global]
>
> fsid = c626be3a-4536-48b9-8db8-470437052313
>
> mon_initial_members = vlsj-kverma1
>
> mon_host = 172.18.84.131
>
> auth_cluster_required = cephx
>
> auth_service_required = cephx
>
> auth_client_required = cephx
>
> # Your network address
>
> public network =
> 172.18.84.0/24<https://urldefense.proofpoint.com/v2/url?u=http-3A__172.18.84.0_24&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=h3q181aKjOAdd1PhqHUiPIOe8moxPTxBm5SaOQp8nyQ&s=A5xvgvnkR4C8tbRyFCVHr22FcL6WmFyuN4lXVZhxP6Y&e=>
>
> osd pool default size = 2
>
> rgw_override_bucket_index_max_shards = 100
>
> debug ms = 1
>
> debug rgw = 20
>
>
>
>
>
> [cephuser@vlsj-kverma1 cluster]$
>
>
>
> [cephuser@vlsj-kverma1 cluster]$ radosgw-admin data sync status
>
> 2019-01-28 10:33:12.163298 7f11c24c79c0 1 Cannot find zone
> id=45c690a8-f39c-4b1d-9faf-e0e991ceaaac (name=san-jose), switching to
> local zonegroup configuration
>
> ERROR: source zone not specified
>
> [cephuser@vlsj-kverma1 cluster]$
>
>
>
> On Secondary site Gateway host:
>
>
>
> [cephuser@zabbix-client ceph]$ cat /etc/ceph/ceph.conf
>
> [global]
>
> fsid = c626be3a-4536-48b9-8db8-470437052313
>
> mon_initial_members = vlsj-kverma1
>
> mon_host = 172.18.84.131
>
> auth_cluster_required = cephx
>
> auth_service_required = cephx
>
> auth_client_required = cephx
>
> # Your network address
>
> public network =
> 172.18.84.0/24<https://urldefense.proofpoint.com/v2/url?u=http-3A__172.18.84.0_24&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=h3q181aKjOAdd1PhqHUiPIOe8moxPTxBm5SaOQp8nyQ&s=A5xvgvnkR4C8tbRyFCVHr22FcL6WmFyuN4lXVZhxP6Y&e=>
>
> osd pool default size = 2
>
> rgw_override_bucket_index_max_shards = 100
>
> debug ms = 1
>
> debug rgw = 20
>
> [client.rgw.zabbix-client]
>
> host = zabbix-client
>
> rgw frontends = "civetweb port=7480"
>
> rgw_zone=san-jose
>
>
>
> [cephuser@zabbix-client ceph]$
>
>
>
>
>
>
>
> Appreciate any help in the setup.
>
>
>
> /Krishna
>
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ceph.com_lis
> tinfo.cgi_ceph-2Dusers-2Dceph.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__tinfo.cgi-5Fceph-2D2Dusers-2D2Dceph.com&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=h3q181aKjOAdd1PhqHUiPIOe8moxPTxBm5SaOQp8nyQ&s=y_ggSZA0AP14vFZc6tA8m5sXOcBU4h4GRRrwqEsKfZU&e=>&d=DwIFaQ&c=aUq983L2pue2FqKFoP6PGHMJQ
> yoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=6MZ
> xZ1DTyLWzj88W8kB9g8C3vLhZvcRI3-Xv_HdQ-Hg&s=_22f69ZqTHaFPKMX21w1VTum2H_
> Nf-44JDmTqQOEH8o&e=
The 'radosgw-admin data sync status' command requires a --source-zone argument,
which is generally the zone name on the opposite cluster.
But you're probably just looking for the 'radosgw-admin sync status'
command, which gives a summary of data and metadata sync against all zones. see
https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.ceph.com_docs_master_radosgw_multisite_-23checking-2Dthe-2Dsync-2Dstatus&d=DwIFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=6MZxZ1DTyLWzj88W8kB9g8C3vLhZvcRI3-Xv_HdQ-Hg&s=3uXnPAw7BleYDrAf-QaRyxvPND_7HZnozAXclhTFbn4&e=
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ceph.com_listinfo.cgi_ceph-2Dusers-2Dceph.com&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=h3q181aKjOAdd1PhqHUiPIOe8moxPTxBm5SaOQp8nyQ&s=kbWd_DF3RmvtVgAJRiHvcBHHMsufKVgVmkpAgYCKiRk&e=>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
