Hi Francois,
       Actually you are discussing two separate questions here:)

1. in the 5 mons(2 in dc1, 2 in dc2, 1 in wan), can the monitor form a quorum? 
How to offload the mon in WAN?
      Yes and No, in one case, you lose any of your DC completely,  that's 
fine, the left  3 monitors could form a quorum.
       But if ONLY the link between DC1 and DC2 is cut (i.e,  the WAN 
connection in both DCs to mon.5 in WAN remains), the problem become kinds of 
tricky since the monitors are splitting minds, mon1 in DC1 and mon 3 in DC2 
have different view in the quorum.  I am not sure if they could reach an 
agreement.
   
       My 0.02 is try to make the VM in WAN as a router? So in the case of link 
between DCs is cut, the monitor could use the spare WAN link to talk with each 
other, not sure.

      The way you use to offload mon.5 is right. Client will randomly pick up 
one monitor(and try other on failure) that provided in the ceph.conf.  So if 
mon.5 doesn't appear in Ceph.conf at all, no client will try to use it.

2. Can the cluster functioning well?
       Well, that depends on your crush_ruleset(how to place the replicas) , 
pool size(how much replicas) and min_size(the minimal success replica for a 
write).
        For example, if your replication rule is: 3 copies, 2 copies in local 
DC and the other 1 copy in geo. Then if you set the pool min_size to 2, the 
cluster can still works with PGs in degraded mode. But if you set min_size to 
3, all the IOs will be blocked there waiting for the replica in geo.

                                                                        Xiaoxi.

-----Original Message-----
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of 
Francois Lafont
Sent: Monday, April 13, 2015 1:04 AM
To: ceph-us...@ceph.com
Subject: [ceph-users] How to dispatch monitors in a multi-site cluster (ie in 2 
datacenters)

Hi,

To summarize, my principal question is: in a ceph cluster, is it possible to 
have, among the monitors, one monitor not necessarily very efficient and with 
potentially network access latencies and still avoid a negative effect on the 
cluster?

I explain the context of my question because it's important. Let's suppose I 
have 2 datacenters: dc1 and dc2. And let's suppose that we can consider the 
network connection between dc1 and dc2 as a real LAN, ie no problem of latency 
between dc1 and dc2 etc. I'm thinking about how to dispatch the monitors 
between dc1 and dc2. Let's suppose I have 5 monitors. I can put 2 monitors on 
dc1 and 3 monitors on dc2. If the connection between dc1 and
dc2 is cut, then the cluster in dc2 will continue to work well because in
dc2 the quorum of monitors is reached but in dc1 the cluster will be stopped 
(no quorum).

Now, what happens if I do this: I put 2 monitors in dc1, 2 monitors on
dc2 and I put the 5th monitors in the WAN, for instance in a VM linked with the 
cluster network by a VPN tunnel (I have one VPN tunnel between
mon.5 and dc1 and one VPN tunnel between mon.5 and dc2). In this case, if the 
connection between dc1 and dc2 is cut (but WAN connection and the VPN tunnels 
in dc1 and dc2 are Ok), in theory the cluster will continue to work in dc1 and 
in dc2 because the quorum is reached (mon.1, mon.2 and
mon.5 in dc1 and mon.3, mon.4 and mon.5 in dc2). Is it correct?

But in this case, how does it work? If a client in dc1 writes data in the OSDs 
of dc1, the data will be not present in the OSD of dc2. It seems to me that 
it's a big problem, unless in fact the cluster does not work in the conditions 
that I've described...

And if the mon.5 is not very efficient with network access latencies, is it a 
problem for the ceph clients? If I just indicate the ip addresses of
mon.1 and mon.2 for the clients in dc1 in the ceph.conf file and if I just 
indicate the ip addresses of mon.3 and mon.4 for the clients in dc2, can I hope 
to avoid the slowness that can generate the mon.5 in the WAN?

Thanks in advance for your help.

--
François Lafont
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to