On Tue, Aug 27, 2013 at 12:30 PM, Francesc Alted <franc...@continuum.io>wrote:
> On Tue, Aug 27, 2013 at 6:25 PM, Alfredo Deza <alfredo.d...@inktank.com>wrote: > >> >> >> >> On Tue, Aug 27, 2013 at 12:04 PM, Francesc Alted >> <franc...@continuum.io>wrote: >> >>> On Tue, Aug 27, 2013 at 5:29 PM, Sage Weil <s...@inktank.com> wrote: >>> >>>> On Tue, 27 Aug 2013, Francesc Alted wrote: >>>> > Hi again, >>>> > >>>> > I continue to try debugging the problem reported before. Now, I have >>>> been >>>> > trying to use a couple of VM for doing this (one with Ubuntu 12.04 >>>> 64-bit, >>>> > and the other with Ubuntu 12.10 64-bit, and I use the ceph.com repos >>>> for >>>> > installing the Ceph libraries). And, unfortunately, I am getting >>>> into the >>>> > same problem: the keyring do not appear where they should (i.e. >>>> > bootstrap-mds and bootstrap-osd in /var/lib/ceph). >>>> > >>>> > I have followed the preflight check list >>>> > (http://ceph.com/docs/next/start/quick-start-preflight/), and the >>>> ceph user >>>> > in the admin box can login perfectly well on the server box, so not >>>> sure >>>> > what's going on here. >>>> > >>>> > I have even tried to use a single ceph server for installing >>>> everything >>>> > (adding the 'osd crush chooseleaf type = 0' line into the ceph conf >>>> file) >>>> > but then again the keyrings do not appear. >>>> > >>>> > Nobody is having the same problems than me (using latest Ceph Dumpling >>>> > 0.67.2 release here)? >>>> > >>>> > Thanks for any insight! >>>> >>>> There are several possible pitfalls here; the missing keys are just the >>>> most visible symptom of the monitors not forming an initial quorum. >>>> >>>> Can you post the contents of your ceph.conf and output from 'ceph daemon >>>> mon.`hostnam` mon-status' on each of the mon nodes? >>>> >>> >>> Okay, I tracked down my problem. It turned out that I was setting >>> different names for the ceph servers in /etc/hosts than their own >>> `hostname`. These log lines when creating the monitor gave me the clue: >>> >>> [ceph-server2][INFO ] creating keyring file: >>> /var/lib/ceph/tmp/ceph-vagrant.mon.keyring >>> [ceph-server2][INFO ] create the monitor keyring file >>> [ceph-server2][INFO ] Running command: ceph-mon --cluster ceph --mkfs >>> -i vagrant --keyring /var/lib/ceph/tmp/ceph-vagrant.mon.keyring >>> [ceph-server2][INFO ] ceph-mon: mon.noname-a 192.168.33.11:6789/0 is >>> local, renaming to mon.vagrant >>> [ceph-server2][INFO ] ceph-mon: set fsid to >>> 253c5a74-699b-44ef-a071-5883716fa620 >>> >>> I was calling this 'vagrant' hostname 'ceph-server2' in my /etc/hosts >>> and I realized this was fooling cephs. So I changed all my /etc/hosts to >>> follow the original hostnames (changed to 'quantal64'), and pum! everything >>> works as intended: >>> >>> [quantal64][INFO ] creating keyring file: >>> /var/lib/ceph/tmp/ceph-quantal64.mon.keyring >>> [quantal64][INFO ] create the monitor keyring file >>> [quantal64][INFO ] Running command: ceph-mon --cluster ceph --mkfs -i >>> quantal64 --keyring /var/lib/ceph/tmp/ceph-quantal64.mon.keyring >>> [quantal64][INFO ] ceph-mon: mon.noname-a 192.168.33.11:6789/0 is >>> local, renaming to mon.quantal64 >>> [quantal64][INFO ] ceph-mon: set fsid to >>> 96c48ec5-7dd5-4f76-81f9-4fdc711a76f0 >>> >>> Now I can gather the keys normally: >>> >>> $ ceph-deploy gatherkeys quantal64 >>> [ceph_deploy.gatherkeys][DEBUG ] Checking quantal64 for >>> /etc/ceph/ceph.client.admin.keyring >>> [ceph_deploy.gatherkeys][DEBUG ] Got ceph.client.admin.keyring key from >>> quantal64. >>> [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring >>> [ceph_deploy.gatherkeys][DEBUG ] Checking quantal64 for >>> /var/lib/ceph/bootstrap-osd/ceph.keyring >>> [ceph_deploy.gatherkeys][DEBUG ] Got ceph.bootstrap-osd.keyring key from >>> quantal64. >>> [ceph_deploy.gatherkeys][DEBUG ] Checking quantal64 for >>> /var/lib/ceph/bootstrap-mds/ceph.keyring >>> [ceph_deploy.gatherkeys][DEBUG ] Got ceph.bootstrap-mds.keyring key from >>> quantal64. >>> >>> Well, thanks anyways. Now it is time to make some more progress and >>> create some ODSs :) >>> >> >> Francesc, thanks for pasting this log info, it is useful to know what >> worked for you :) I will update the docs for ceph-deploy >> on things to watch out so that there is *something* users can try when >> this comes up. >> > > No problem. A possible idea for enhancing the capabilities to > self-detecting problems would be to implement a check in ceph-deploy (or in > another place) that warns (or just gives an error) when it detects that the > hostname is different depending on whether they do a DNS lookup or a > `hostname` output. > I went ahead and created http://tracker.ceph.com/issues/6132 to track this. Thanks again. > > -- > Francesc Alted
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
