Thanks Greg - I thought the same thing, but confirmed with the user that it appears the radosgw client is indeed using initial members - when he added all of his hosts to initial members, things worked just fine. In either event, all of the monitors were always fully enumerated later in the config file. Is this potentially a bug specific to radosgw? Here's his config file:
[global] fsid = fc0e2e09-ade3-4ff6-b23e-f789775b2515 mon initial members = nodo-3 auth cluster required = cephx auth service required = cephx auth client required = cephx osd pool default size = 3 osd pool default min_size = 1 osd pool default pg_num = 128 osd pool default pgp_num = 128 osd recovery delay start = 15 log file = /dev/stdout mon_clock_drift_allowed = 1 [mon.nodo-1] host = nodo-1 mon addr = 192.168.2.200:6789 [mon.nodo-2] host = nodo-2 mon addr = 192.168.2.201:6789 [mon.nodo-3] host = nodo-3 mon addr = 192.168.2.202:6789 [client.radosgw.gateway] host = deis-store-gateway keyring = /etc/ceph/ceph.client.radosgw.keyring rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock log file = /dev/stdout On Wed, Dec 10, 2014 at 11:40 AM, Gregory Farnum <g...@gregs42.com> wrote: > On Tue, Dec 9, 2014 at 3:11 PM, Christopher Armstrong > <ch...@opdemand.com> wrote: > > Hi folks, > > > > I think we have a bit of confusion around how initial members is used. I > > understand that we can specify a single monitor (or a subset of > monitors) so > > that the cluster can form a quorum when it first comes up. This is how > we're > > using the setting now - so the cluster can come up with just one monitor, > > with the other monitors to follow later. > > > > However, a Deis user reported that when the monitor in his initial > members > > list went down, radosgw stopped functioning, even though there are three > > mons in his config file. I would think that the radosgw client would > connect > > to any of the nodes in the config file to get the state of the cluster, > and > > that the initial members list is only used when the monitors first come > up > > and are trying to achieve quorum. > > > > The issue he filed is here: https://github.com/deis/deis/issues/2711 > > > > He also found this Ceph issue filed: > https://github.com/ceph/ceph/pull/1233 > > Nope, this has nothing to do with it. > > > > > Is that what we're seeing here? Can anyone point us in the right > direction? > > I didn't see the actual conf file posted anywhere to look at, but my > guess is simply that (since it looks like you're using generated conf > files which can differ across hosts) that the one on the server(s) in > question don't have the monitors listed in them. I'm only skimming the > code, but from it and my recollection, when a Ceph client starts up it > will try to assemble a list of monitors to contact from: > 1) the contents of the "mon host" config entry > 2) the "mon addr" value in any of the "global", "mon" or "mon.X" sections > > The clients don't even look at mon_initial_members that I can see, > actually — so perhaps your client config only lists the initial > monitor, without adding the others? > -Greg >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
