On Fri, Apr 4, 2014 at 2:53 AM, Diedrich Ehlerding
<diedrich.ehlerd...@ts.fujitsu.com> wrote:
> I am trying to deploy a cluster with ceph-deploy. I installed ceph
> 0.72.2 from the rpm repositories. Running "ceph-deploy mon
> create-initial" creates /var/lib/ceph etc. on all the nodes, but on
> all nodes I get a warning:
>
>
> [hvrrzceph2][DEBUG ] Starting Ceph mon.hvrrzceph2 on hvrrzceph2...
> [hvrrzceph2][DEBUG ] Starting ceph-create-keys on hvrrzceph2...
> [hvrrzceph2][WARNING] No data was received after 7 seconds,
> disconnecting...
>
>
> and afterwards, ceph -s cannot connect to the cluster. No
> client.admin keyring is created in ceph.conf.
>
> Then I attempted to create one monitor only on one node. Again, no
> keys for client.admin, bootstrap-mds and bootstrap-osd were created.
> Here is the complete log of this attempt:
>
> hvrrzceph1:~/my-cluster # ceph-deploy new hvrrzceph1
> [ceph_deploy.cli][INFO ] Invoked (1.4.0): /usr/bin/ceph-deploy new
> hvrrzceph1
> [ceph_deploy.new][DEBUG ] Creating new cluster named ceph
> [ceph_deploy.new][DEBUG ] Resolving host hvrrzceph1
> [ceph_deploy.new][DEBUG ] Monitor hvrrzceph1 at 10.1.1.239
> [ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
> [ceph_deploy.new][DEBUG ] Monitor initial members are ['hvrrzceph1']
> [ceph_deploy.new][DEBUG ] Monitor addrs are ['10.1.1.239']
> [ceph_deploy.new][DEBUG ] Creating a random mon key...
> [ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
> [ceph_deploy.new][DEBUG ] Writing monitor keyring to
> ceph.mon.keyring...
> hvrrzceph1:~/my-cluster # vi ceph.conf
>
> [added "cluster network" + "public network"; leaving them away doesnt
> change anything]
>
> hvrrzceph1:~/my-cluster # cat ceph.conf
> [global]
> auth_service_required = cephx
> filestore_xattr_use_omap = true
> auth_client_required = cephx
> auth_cluster_required = cephx
> mon_host = 10.1.1.239
> mon_initial_members = hvrrzceph1
> fsid = ec73a230-f645-456f-9523-9a03621d18dd
> cluster_network = 10.112.0.0/16
> public_network = 10.111.0.0/16
>
>
> hvrrzceph1:~/my-cluster # ceph-deploy mon create-initial
> [ceph_deploy.cli][INFO ] Invoked (1.4.0): /usr/bin/ceph-deploy mon
> create-initial
> [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts
> hvrrzceph1
> [ceph_deploy.mon][DEBUG ] detecting platform for host hvrrzceph1 ...
> [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
> [hvrrzceph1][DEBUG ] detect platform information from remote host
> [hvrrzceph1][DEBUG ] detect machine type
> [ceph_deploy.mon][INFO ] distro info: SUSE Linux Enterprise Server
> 11 x86_64
> [hvrrzceph1][DEBUG ] determining if provided host has same hostname
> in remote
> [hvrrzceph1][DEBUG ] get remote short hostname
> [hvrrzceph1][DEBUG ] deploying mon to hvrrzceph1
> [hvrrzceph1][DEBUG ] get remote short hostname
> [hvrrzceph1][DEBUG ] remote hostname: hvrrzceph1
> [hvrrzceph1][DEBUG ] write cluster configuration to
> /etc/ceph/{cluster}.conf
> [hvrrzceph1][DEBUG ] create the mon path if it does not exist
> [hvrrzceph1][DEBUG ] checking for done path:
> /var/lib/ceph/mon/ceph-hvrrzceph1/done
> [hvrrzceph1][DEBUG ] done path does not exist:
> /var/lib/ceph/mon/ceph-hvrrzceph1/done
> [hvrrzceph1][INFO ] creating keyring file:
> /var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring
> [hvrrzceph1][DEBUG ] create the monitor keyring file
> [hvrrzceph1][INFO ] Running command: ceph-mon --cluster ceph --mkfs
> -i hvrrzceph1 --keyring /var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring
> [hvrrzceph1][DEBUG ] ceph-mon: set fsid to
> ec73a230-f645-456f-9523-9a03621d18dd
> [hvrrzceph1][DEBUG ] ceph-mon: created monfs at
> /var/lib/ceph/mon/ceph-hvrrzceph1 for mon.hvrrzceph1
> [hvrrzceph1][INFO ] unlinking keyring file
> /var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring
> [hvrrzceph1][DEBUG ] create a done file to avoid re-doing the mon
> deployment
> [hvrrzceph1][DEBUG ] create the init path if it does not exist
> [hvrrzceph1][INFO ] Running command: rcceph -c /etc/ceph/ceph.conf
> start mon.hvrrzceph1
> [hvrrzceph1][DEBUG ] === mon.hvrrzceph1 ===
> [hvrrzceph1][DEBUG ] Starting Ceph mon.hvrrzceph1 on hvrrzceph1...
> [hvrrzceph1][DEBUG ] Starting ceph-create-keys on hvrrzceph1...
> [hvrrzceph1][WARNIN] No data was received after 7 seconds,
> disconnecting...
> [hvrrzceph1][INFO ] Running command: ceph --cluster=ceph
> --admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status
> [hvrrzceph1][DEBUG ]
> **********************************************************************
> **********
> [hvrrzceph1][DEBUG ] status for monitor: mon.hvrrzceph1
> [hvrrzceph1][DEBUG ] {
> [hvrrzceph1][DEBUG ] "election_epoch": 2,
> [hvrrzceph1][DEBUG ] "extra_probe_peers": [
> [hvrrzceph1][DEBUG ] "10.1.1.239:6789/0"
> [hvrrzceph1][DEBUG ] ],
> [hvrrzceph1][DEBUG ] "monmap": {
> [hvrrzceph1][DEBUG ] "created": "0.000000",
> [hvrrzceph1][DEBUG ] "epoch": 1,
> [hvrrzceph1][DEBUG ] "fsid":
> "ec73a230-f645-456f-9523-9a03621d18dd",
> [hvrrzceph1][DEBUG ] "modified": "0.000000",
> [hvrrzceph1][DEBUG ] "mons": [
> [hvrrzceph1][DEBUG ] {
> [hvrrzceph1][DEBUG ] "addr": "10.111.3.1:6789/0",
> [hvrrzceph1][DEBUG ] "name": "hvrrzceph1",
> [hvrrzceph1][DEBUG ] "rank": 0
> [hvrrzceph1][DEBUG ] }
> [hvrrzceph1][DEBUG ] ]
> [hvrrzceph1][DEBUG ] },
> [hvrrzceph1][DEBUG ] "name": "hvrrzceph1",
> [hvrrzceph1][DEBUG ] "outside_quorum": [],
> [hvrrzceph1][DEBUG ] "quorum": [
> [hvrrzceph1][DEBUG ] 0
> [hvrrzceph1][DEBUG ] ],
> [hvrrzceph1][DEBUG ] "rank": 0,
> [hvrrzceph1][DEBUG ] "state": "leader",
> [hvrrzceph1][DEBUG ] "sync_provider": []
> [hvrrzceph1][DEBUG ] }
> [hvrrzceph1][DEBUG ]
> **********************************************************************
> **********
> [hvrrzceph1][INFO ] monitor: mon.hvrrzceph1 is running
> [hvrrzceph1][INFO ] Running command: ceph --cluster=ceph
> --admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status
> [ceph_deploy.mon][INFO ] processing monitor mon.hvrrzceph1
> [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
> [hvrrzceph1][INFO ] Running command: ceph --cluster=ceph
> --admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status
> [ceph_deploy.mon][INFO ] mon.hvrrzceph1 monitor has reached quorum!
> [ceph_deploy.mon][INFO ] all initial monitors are running and have
> formed quorum
> [ceph_deploy.mon][INFO ] Running gatherkeys...
> [ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for
> /etc/ceph/ceph.client.admin.keyring
> [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
> [hvrrzceph1][DEBUG ] detect platform information from remote host
> [hvrrzceph1][DEBUG ] detect machine type
> [hvrrzceph1][DEBUG ] fetch remote file
> [ceph_deploy.gatherkeys][WARNIN] Unable to find
> /etc/ceph/ceph.client.admin.keyring on ['hvrrzceph1']
> [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
> [ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for
> /var/lib/ceph/bootstrap-osd/ceph.keyring
> [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
> [hvrrzceph1][DEBUG ] detect platform information from remote host
> [hvrrzceph1][DEBUG ] detect machine type
> [hvrrzceph1][DEBUG ] fetch remote file
> [ceph_deploy.gatherkeys][WARNIN] Unable to find
> /var/lib/ceph/bootstrap-osd/ceph.keyring on ['hvrrzceph1']
> [ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for
> /var/lib/ceph/bootstrap-mds/ceph.keyring
> [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
> [hvrrzceph1][DEBUG ] detect platform information from remote host
> [hvrrzceph1][DEBUG ] detect machine type
> [hvrrzceph1][DEBUG ] fetch remote file
> [ceph_deploy.gatherkeys][WARNIN] Unable to find
> /var/lib/ceph/bootstrap-mds/ceph.keyring on ['hvrrzceph1']
> hvrrzceph1:~/my-cluster #
>
> I had already a running cluster, and ceph-deploy had worked ... :-(
> Any hints where I can find what is missing?
Have you ensured that either there is no firewall up or that the ports
that the monitors need to communicate
between each other are open?
If that is not the problem then the next thing I would do is to
increase the verbosity for the monitors, restart them
and look at the logs.
>
> TIA.
> --
> Diedrich Ehlerding, Fujitsu Technology Solutions GmbH,
> FTS CE SC PS&IS W, Hildesheimer Str 25, D-30880 Laatzen
> Fon +49 511 8489-1806, Fax -251806, Mobil +49 173 2464758
> Firmenangaben: http://de.ts.fujitsu.com/imprint.html
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
