Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread Mandell Degerness
Precisely. This is what we need to do. It is just a case of adjusting our process to make that possible. As I stated a couple e-mails ago, the design of Ceph allows it, it is just a bit of a challenge to fit it into our existing processes. It's on me now to fix the process. On Mon, Jun 24, 201

Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread Alex Bligh
On 25 Jun 2013, at 00:39, Mandell Degerness wrote: > The issue, Sage, is that we have to deal with the cluster being > re-expanded. If we start with 5 monitors and scale back to 3, running > the "ceph mon remove N" command after stopping each monitor and don't > restart the existing monitors, we

Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread Mandell Degerness
The issue, Sage, is that we have to deal with the cluster being re-expanded. If we start with 5 monitors and scale back to 3, running the "ceph mon remove N" command after stopping each monitor and don't restart the existing monitors, we cannot re-add those same monitors that were previously remov

Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread Sage Weil
On Mon, 24 Jun 2013, Mandell Degerness wrote: > Hmm. This is a bit ugly from our perspective, but not fatal to your > design (just our implementation). At the time we run the rm, the > cluster is smaller and so the restart of each monitor is not fatal to > the cluster. The problem is on our side

Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread Mandell Degerness
Hmm. This is a bit ugly from our perspective, but not fatal to your design (just our implementation). At the time we run the rm, the cluster is smaller and so the restart of each monitor is not fatal to the cluster. The problem is on our side in terms of guaranteeing order of behaviors. On Mon,

Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread Sage Weil
On Mon, 24 Jun 2013, Mandell Degerness wrote: > I'm testing the change (actually re-starting the monitors after the > monitor removal), but this brings up the issue with why we didn't want > to do this in the first place: When reducing the number of monitors > from 5 to 3, we are guaranteed to hav

Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread Mandell Degerness
I'm testing the change (actually re-starting the monitors after the monitor removal), but this brings up the issue with why we didn't want to do this in the first place: When reducing the number of monitors from 5 to 3, we are guaranteed to have a service outage for the time it takes to restart at

Re: [ceph-users] monitor removal and re-add

2013-06-24 Thread John Nielsen
On Jun 21, 2013, at 5:00 PM, Mandell Degerness wrote: > There is a scenario where we would want to remove a monitor and, at a > later date, re-add the monitor (using the same IP address). Is there > a supported way to do this? I tried deleting the monitor directory > and rebuilding from scratch

[ceph-users] monitor removal and re-add

2013-06-21 Thread Mandell Degerness
There is a scenario where we would want to remove a monitor and, at a later date, re-add the monitor (using the same IP address). Is there a supported way to do this? I tried deleting the monitor directory and rebuilding from scratch following the add monitor procedures from the web, but the moni