Hi, noted and thanks a lot.
Best Rgds
/stwong
-----Original Message-----
From: Ricardo Dias <rd...@suse.com>
Sent: Thursday, July 25, 2019 8:47 PM
To: ST Wong (ITSC) <s...@itsc.cuhk.edu.hk>; Manuel Lausch
<manuel.lau...@1und1.de>; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] Please help: change IP address of a cluster
Hi,
The monmaptool has a different option to add an address vector. To add an
address vector do this:
monmaptool --addv node1 [v2:10.0.1.1:3330,v1:10.0.1.1:6789] {tmp}/{filename}
The --addv option does not appear in the usage description but the fix is
already under way: https://github.com/ceph/ceph/pull/29307
Cheers,
--
Ricardo Dias
Senior Software Engineer - Storage Team
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284
(AG Nürnberg)
On 24/07/19 07:20, ST Wong (ITSC) wrote:
> Hi,
>
> Thanks for your help.
> I changed IP addresses of OSD nodes successfully. When changing IP address
> on MON, it works except that the MON only listens on v2 port 3300 after
> adding the MON back to the cluster. Previously the MON listens on both v1
> (6789) and v2 (3300).
> Besides, can't add both v1 and v2 entries manually using monmaptool like
> following. Only the 2nd one will get added.
>
> monmaptool -add node1 v1:10.0.1.1:6789, v2:10.0.1.1:3330
> {tmp}/{filename}
>
> the monmap now looks like following:
>
> min_mon_release 14 (nautilus)
> 0: [v2:10.0.1.92:3300/0,v1:10.0.1.92:6789/0] mon.cmon2
> 1: [v2:10.0.1.93:3300/0,v1:10.0.1.93:6789/0] mon.cmon2
> 2: [v2:10.0.1.94:3300/0,v1:10.0.1.94:6789/0] mon.cmon3
> 3: [v2:10.0.1.95:3300/0,v1:10.0.1.95:6789/0] mon.cmon4
> 4: v2:10.0.1.97:3300/0 mon.cmon5 <--- the MON
> being removed/added
>
> Although it's okay to use v2 only, I'm afraid I've missed some steps and
> messed the cluster up. Any advice?
> Thanks again.
>
> Best Regards,
> /stwong
>
> -----Original Message-----
> From: ceph-users <ceph-users-boun...@lists.ceph.com> On Behalf Of
> Manuel Lausch
> Sent: Tuesday, July 23, 2019 7:32 PM
> To: ceph-users@lists.ceph.com
> Subject: Re: [ceph-users] Please help: change IP address of a cluster
>
> Hi,
>
> I had to change the IPs of my cluster some time ago. The process was quite
> easy.
>
> I don't understand what you mean with configuring and deleting static
> routes. The easies way is if the router allows (at least for the
> change) all traffic between the old and the new network.
>
> I did the following steps.
>
> 1. Add the new IP Network space separated to the "public network" line
> in your ceph.conf
>
> 2. OSDS: stop you OSDs on the first node. Reconfigure the host network
> and start your OSDs again. Repeat this for all hosts one by one
>
> 3. MON: stop and remove one mon from cluster, delete all data in
> /var/ceph/mon/mon.<id> reconfigure the host network. Create the new mon
> instance (don't forget the "mon host" entrys in your ceph.conf and your
> clients as well) Of course this requires at least 3 Mons in your cluster!
> After 2 of 5 Mons in my cluster I added the new mon adresses to my clients
> and restarted them.
>
> 4. MGR: stop the mgr daemon. reconfigure the host network. Start the
> mgr daemon one by one
>
>
> I wouldn't recomend the "messy way" to reconfigure your mons. removing and
> adding mons to the cluster is quite easy and in my opinion the most secure.
>
> The complet IP change in our cluster worked without outage while the cluster
> was in production.
>
> I hope I could help you.
>
> Regards
> Manuel
>
>
>
> On Fri, 19 Jul 2019 10:22:37 +0000
> "ST Wong (ITSC)" <s...@itsc.cuhk.edu.hk> wrote:
>
>> Hi all,
>>
>> Our cluster has to change to new IP range in same VLAN: 10.0.7.0/24
>> -> 10.0.18.0/23, while IP address on private network for OSDs
>> remains unchanged. I wonder if we can do that in either one following
>> ways:
>>
>> =====================
>>
>> 1.
>>
>> a. Define static route for 10.0.18.0/23 on each node
>>
>> b. Do it one by one:
>>
>> For each monitor/mgr:
>>
>> - remove from cluster
>>
>> - change IP address
>>
>> - add static route to original IP range 10.0.7.0/24
>>
>> - delete static route for 10.0.18.0/23
>>
>> - add back to cluster
>>
>> For each OSD:
>>
>> - stop OSD daemons
>>
>> - change IP address
>>
>> - add static route to original IP range 10.0.7.0/24
>>
>> - delete static route for 10.0.18.0/23
>>
>> - start OSD daemons
>>
>> c. Clean up all static routes defined.
>>
>>
>>
>> 2.
>>
>> a. Export and update monmap using the messy way as described in
>> http://docs.ceph.com/docs/mimic/rados/operations/add-or-rm-mons/
>>
>>
>>
>> ceph mon getmap -o {tmp}/{filename}
>>
>> monmaptool -rm node1 -rm node2 ... --rm node n {tmp}/{filename}
>>
>> monmaptool -add node1 v2:10.0.18.1:3330,v1:10.0.18.1:6789 -add node2
>> v2:10.0.18.2:3330,v1:10.0.18.2:6789 ... --add nodeN
>> v2:10.0.18.N:3330,v1:10.0.18.N:6789 {tmp}/{filename}
>>
>>
>>
>> b. stop entire cluster daemons and change IP addresses
>>
>>
>> c. For each mon node: ceph-mon -I {mon-id} -inject-monmap
>> {tmp}/{filename}
>>
>>
>>
>> d. Restart cluster daemons.
>>
>>
>>
>> 3. Or any better method...
>> =====================
>>
>> Would anyone please help? Thanks a lot.
>> Rgds
>> /st wong
>>
>
>
>
> --
> Manuel Lausch
>
> Systemadministrator
> Storage Services
>
> 1&1 Mail & Media Development & Technology GmbH | Brauerstraße 48 |
> 76135 Karlsruhe | Germany Phone: +49 721 91374-1847
> E-Mail: manuel.lau...@1und1.de | Web: www.1und1.de
>
> Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 5452
>
> Geschäftsführer: Alexander Charles, Thomas Ludwig, Jan Oetjen, Sascha
> Vollmer
>
>
> Member of United Internet
>
> Diese E-Mail kann vertrauliche und/oder gesetzlich geschützte Informationen
> enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind oder diese
> E-Mail irrtümlich erhalten haben, unterrichten Sie bitte den Absender und
> vernichten Sie diese E-Mail. Anderen als dem bestimmungsgemäßen Adressaten
> ist untersagt, diese E-Mail zu speichern, weiterzuleiten oder ihren Inhalt
> auf welche Weise auch immer zu verwenden.
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient of this e-mail, you are hereby notified that
> saving, distribution or use of the content of this e-mail in any way is
> prohibited. If you have received this e-mail in error, please notify the
> sender and delete the e-mail.
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
