Petr, My name is Zac Dover. I'm the upstream docs guy for the Ceph Foundation.
I will begin the process of correcting this part of the documentation. I will begin by reviewing the section "Creating a Secondary Zone". My schedule is full until Sunday, but I will raise an issue in tracker.ceph.com on Sunday and I will keep you apprised of the progress. Zac Dover Upstream Docs Ceph Foundation ------- Original Message ------- On Thursday, August 31st, 2023 at 12:31 AM, Petr Bena <petr@bena.rocks> wrote: > > > Hello, > > My goal is to setup multisite RGW with 2 separate CEPH clusters in separate > datacenters, where RGW data are being replicated. I created a lab for this > purpose in both locations (with latest reef ceph installed using cephadm) and > tried to follow this guide: https://docs.ceph.com/en/reef/radosgw/multisite/ > > Unfortunatelly, even after multiple attempts it always failed when creating a > secondary zone. I could succesfully pull the realm from master, but that was > pretty much the last trully succesful step. I can notice that immediately > after pulling the realm to secondary, radosgw-admin user list returns an > empty list (which IMHO should contain replicated user list from master). > Continuing by setting default real and zonegroup and creating the secondary > zone in secondary cluster I end up having 2 zones in each cluster, both > seemingly in same zonegroup, but with replication failing - this is what I > see in sync status: > > (master) [ceph: root@ceph-lab-brn-01 /]# radosgw-admin sync status > realm d2c4ebf9-e156-4c4e-9d56-3fff6a652e75 (ceph) > zonegroup abc3c0ae-a84d-48d4-8e78-da251eb78781 (cz) > zone 97fb5842-713a-4995-8966-5afe1384f17f (cz-brn) > current time 2023-08-30T12:58:12Z > zonegroup features enabled: resharding > disabled: compress-encrypted > metadata sync no sync (zone is master) > 2023-08-30T12:58:13.991+0000 7f583a52c780 0 ERROR: failed to fetch datalog > info > data sync source: 13a8c663-b241-4d8a-a424-8785fc539ec5 (cz-hol) > failed to retrieve sync info: (13) Permission denied > > > (secondary) [ceph: root@ceph-lab-hol-01 /]# radosgw-admin sync status > realm d2c4ebf9-e156-4c4e-9d56-3fff6a652e75 (ceph) > zonegroup abc3c0ae-a84d-48d4-8e78-da251eb78781 (cz) > zone 13a8c663-b241-4d8a-a424-8785fc539ec5 (cz-hol) > current time 2023-08-30T12:58:54Z > zonegroup features enabled: resharding > disabled: compress-encrypted > metadata sync failed to read sync status: (2) No such file or directory > 2023-08-30T12:58:55.617+0000 7ff37c9db780 0 ERROR: failed to fetch datalog > info > data sync source: 97fb5842-713a-4995-8966-5afe1384f17f (cz-brn) > failed to retrieve sync info: (13) Permission denied > > > In master there is one user created during the process > (synchronization-user), on slave there are no users and when I try to > re-create this synchronization user it complains I shouldn't even try and > instead execute the command on master. I can see same realm and zonegroup IDs > on both sides, zone list is different though: > > > (master) [ceph: root@ceph-lab-brn-01 /]# radosgw-admin zone list > { > "default_info": "97fb5842-713a-4995-8966-5afe1384f17f", > "zones": [ > "cz-brn", > "default" > ] > } > > > (secondary) [ceph: root@ceph-lab-hol-01 /]# radosgw-admin zone list > { > "default_info": "13a8c663-b241-4d8a-a424-8785fc539ec5", > "zones": [ > "cz-hol", > "default" > ] > } > > > The permission denied error is puzzling me - could it be because real pull > didn't sync the users? I tried this multiple times with clean ceph install on > both sides - and always ended up the same. I even tried force creating the > same user with same secrets on the other side, but it didn't help. How can I > debug what kind of secret is secondary trying to use when communicating with > master? Could it be that this multisite RGW setup is not yet truly supported > in reef? I noticed that the documentation itself seems written for older ceph > versions, as there are no mentions about orchestrator (for example in steps > where configuration files of RGW need to be edited, which is done differently > when using cephadm). > > I think that documentation is simply wrong at this time. Either it's missing > some crucial steps, or it's outdated or otherwise unclear - simply by > following all the steps as outlined there, you are likely to end up the same. > > > Thanks for help! > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
