Here is the procedure I wrote for our internal use (it is still a work in
progress) and may help you:
*Creating the First Monitor*
Once you Ceph installed, DNS and networking configured and have a ceph.conf
file built, you are ready to bootstrap the first monitor. The UUID is the
same from the ceph.conf file generated earlier, cluster-name is the name of
the Ceph cluster, usually just 'ceph', hostname is the short name of the
host and must match `hostname -s`.
1. SSH into the monitor host
2. Create the monitor keyring
ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon.
--cap mon 'allow *'
3. Create the admin keyring
ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring
--gen-key -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow
*' --cap mds 'allow'
4. Add the admin key to the monitor keyring so the Admin user can manage
the cluster
ceph-authtool /tmp/ceph.mon.keyring --import-keyring
/etc/ceph/ceph.client.admin.keyring
5. Create the initial monitor map
monmaptool --create --add {hostname} {ip-address} --fsid {uuid} /tmp/monmap
6. Create the directory the monitor will store persistent data
sudo mkdir /var/lib/ceph/mon/{cluster-name}-{hostname}
7. Create the file structure for the monitor
ceph-mon --mkfs -i {hostname} --monmap /tmp/monmap --keyring
/tmp/ceph.mon.keyring
8. Let the monitor know that everything is ready to go
sudo touch /var/lib/ceph/mon/{cluster-name}-{hostname}/done
9. Start the Monitor service
sudo start ceph-mon id=node1
10. Set the Monitor service to start at boot
*Adding Additional Monitors*
Adding additional monitors will make the cluster less susceptible to
outages due to hardware running the monitors going offline. As monitors are
added, the load on each monitor can both increase and decrease, so it is
import to understand when a new set of monitors should be added (to
understand why two monitors should be added, please see Monitors). Having
more monitors reduces the number of clients that each monitor has to
service. However, each time a CRUSH map changes (an OSD is marked out, a
new OSD is added, etc), a majority of monitors have to agree on the CRUSH
changes in the PAXOS algorithm and the CRUSH map has to be updated on each
Monitor. If the cluster is experiencing many CRUSH map changes it can add
additional load on the Monitors.
Monitors perform many file sync operations and are sensitive to latencies
with these operations. The large number of file sync operations can be very
disruptive to to OSD processes especially if residing on the same
traditional rotational disk. It is best to have a Monitor on its own
dedicated hardware. If dedicated hardware is not an option, then locate the
Monitor store on an SSD that is not primarily used for an OSD, an SSD
journal would be OK.
The process of adding a new monitor is detailed in (
http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#adding-a-monitor-manual)
and is outlined as follows:
1. Copy the Monitor key and Monitor map from a running Monitor to the
new monitor.
2. Create a monitor directory on the new monitor.
3. Add the new monitor to the Monitor map.
4. Start the new monitor.
On Thu, Mar 12, 2015 at 9:58 AM, Jesus Chavez (jeschave) <jesch...@cisco.com
> wrote:
> Hi Robert yes I did disable completely actually with chkconfig off for
> not take the service up when booting, I have 2 networks 1 with internet for
> yum purposes and the network for the public network so before any
> configuration I specified on ceph.conf that public network but I am not
> sure if it is the cause for something...
> The thing is that I am not sure about which steps should run in the new
> monitor host and which one should I run in the initial monitor, it seems
> like step 3 and 4 that is generate keyring and mapping should be done in
> initial monitor server and also step 5 that is mkfs becuase if I try to run
> those steps in the new monitor host didnt work cause cant find the keys :(
>
>
> * Jesus Chavez*
> SYSTEMS ENGINEER-C.SALES
>
> jesch...@cisco.com
> Phone: *+52 55 5267 3146 <+52%2055%205267%203146>*
> Mobile: *+51 1 5538883255 <+51%201%205538883255>*
>
> CCIE - 44433
>
> On Mar 12, 2015, at 7:54 AM, Robert LeBlanc <rob...@leblancnet.us> wrote:
>
> If I remember right, the mon key has to be the same between all the mon
> hosts. I don't think I added an admin key to my second mon, it got all the
> other keys once it joined the mon closure. I do remember the join taking a
> while. Have you checked the firewall to make sure traffic is allowed? I
> don't remember if you said you checked it.
>
> Robert LeBlanc
>
> Sent from a mobile device please excuse any typos.
> On Mar 11, 2015 8:08 PM, "Jesus Chavez (jeschave)" <jesch...@cisco.com>
> wrote:
>
>> Thanks Steffen I have followed everything not sure what is going on,
>> the mon keyring and client admin are individual? Per mon host? Or do I need
>> to copy from the first initial mon node?
>>
>> Thanks again!
>>
>>
>> * Jesus Chavez*
>> SYSTEMS ENGINEER-C.SALES
>>
>> jesch...@cisco.com
>> Phone: *+52 55 5267 3146 <+52%2055%205267%203146>*
>> Mobile: *+51 1 5538883255 <+51%201%205538883255>*
>>
>> CCIE - 44433
>>
>> On Mar 11, 2015, at 6:28 PM, Steffen W Sørensen <ste...@me.com> wrote:
>>
>>
>> On 12/03/2015, at 00.55, Jesus Chavez (jeschave) <jesch...@cisco.com>
>> wrote:
>>
>> can anybody tell me a good blog link that explain how to add monitor? I
>> have tried manually and also with ceph-deploy without success =(
>>
>> Dunno if these might help U:
>>
>>
>> http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#adding-a-monitor-manual
>>
>> http://cephnotes.ksperis.com/blog/2013/08/29/mon-failed-to-start
>>
>> /Steffen
>>
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
