Re: [ceph-users] Disaster recovery of monitor

2015-11-17 Thread Jose Tavares
Hi guys ... Thanks a lot for your support. I discovered what happened. I had 2 monitors, osnode01 and osnode02. I tried do add a 3rd by using ceph-deploy. ceph-deploy was using a key different from the one used by my monitor cluster. So, I added osnode08 to the monitor cluster and it did not be

Re: [ceph-users] Disaster recovery of monitor

2015-11-17 Thread Jose Tavares
Now I tried to inject the latest map I had. Also, I created a second monitor on osnode02, like I had before, using the same map. I started both monitors ... Logs from osnode01 show my content ... and then it started to show lines like 2015-11-17 10:56:26.515069 7fc73af67700 0 mon.osnode01@0(prob

Re: [ceph-users] Disaster recovery of monitor

2015-11-17 Thread Joao Eduardo Luis
On 11/17/2015 12:27 PM, Jose Tavares wrote: > My concern is about this log line > > 2015-11-17 10:11:16.143864 7f81e14aa700 0 > mon.osnode01@0(probing).data_health(0) update_stats avail 19% total 220 > GB, used 178 GB, avail 43194 MB > > I use to have 7TB of available space with 263G of con

Re: [ceph-users] Disaster recovery of monitor

2015-11-17 Thread Jose Tavares
On Tue, Nov 17, 2015 at 7:27 AM, Joao Eduardo Luis wrote: > On 11/17/2015 03:56 AM, Jose Tavares wrote: > > The problem is that I think I don't have any good monitor anymore. > > How do I know if the map I am trying is ok? > > > > I also saw in the logs that the primary mon was trying to contact

Re: [ceph-users] Disaster recovery of monitor

2015-11-17 Thread Jose Tavares
On Tue, Nov 17, 2015 at 6:32 AM, Wido den Hollander wrote: > On 11/17/2015 04:56 AM, Jose Tavares wrote: > > The problem is that I think I don't have any good monitor anymore. > > How do I know if the map I am trying is ok? > > > > How do you mean there is no good monitor? Did you encounter a dis

Re: [ceph-users] Disaster recovery of monitor

2015-11-17 Thread Joao Eduardo Luis
On 11/17/2015 03:56 AM, Jose Tavares wrote: > The problem is that I think I don't have any good monitor anymore. > How do I know if the map I am trying is ok? > > I also saw in the logs that the primary mon was trying to contact a > removed mon at IP .112 .. So, I added .112 again ... and it didn'

Re: [ceph-users] Disaster recovery of monitor

2015-11-17 Thread Wido den Hollander
On 11/17/2015 04:56 AM, Jose Tavares wrote: > The problem is that I think I don't have any good monitor anymore. > How do I know if the map I am trying is ok? > How do you mean there is no good monitor? Did you encounter a disk failure or something? > I also saw in the logs that the primary mon

Re: [ceph-users] Disaster recovery of monitor

2015-11-16 Thread Jose Tavares
The problem is that I think I don't have any good monitor anymore. How do I know if the map I am trying is ok? I also saw in the logs that the primary mon was trying to contact a removed mon at IP .112 .. So, I added .112 again ... and it didn't help. Attached are the logs of what is going on and

[ceph-users] Disaster recovery of monitor

2015-11-16 Thread Jose Tavares
Hi guys ... I need some help as my cluster seems to be corrupted. I saw here .. https://www.mail-archive.com/ceph-users@lists.ceph.com/msg01919.html .. a msg from 2013 where Peter had a problem with his monitors. I had the same problem today when trying to add a new monitor, and than playing with

Re: [ceph-users] Disaster recovery of monitor

2013-06-17 Thread peter
On 2013-06-14 19:59, Joao Eduardo Luis wrote: On 06/14/2013 02:39 PM, pe...@2force.nl wrote: On 2013-06-13 20:10, pe...@2force.nl wrote: On 2013-06-13 18:57, Joao Eduardo Luis wrote: On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 1

Re: [ceph-users] Disaster recovery of monitor

2013-06-14 Thread Joao Eduardo Luis
On 06/14/2013 02:39 PM, pe...@2force.nl wrote: On 2013-06-13 20:10, pe...@2force.nl wrote: On 2013-06-13 18:57, Joao Eduardo Luis wrote: On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 13, 2013, wrote: Hello, We ran into a problem

Re: [ceph-users] Disaster recovery of monitor

2013-06-14 Thread peter
On 2013-06-14 16:38, Joao Eduardo Luis wrote: On 06/14/2013 02:39 PM, pe...@2force.nl wrote: On 2013-06-13 20:10, pe...@2force.nl wrote: On 2013-06-13 18:57, Joao Eduardo Luis wrote: On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 1

Re: [ceph-users] Disaster recovery of monitor

2013-06-14 Thread Joao Eduardo Luis
On 06/14/2013 02:39 PM, pe...@2force.nl wrote: On 2013-06-13 20:10, pe...@2force.nl wrote: On 2013-06-13 18:57, Joao Eduardo Luis wrote: On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 13, 2013, wrote: Hello, We ran into a problem

Re: [ceph-users] Disaster recovery of monitor

2013-06-14 Thread Joao Eduardo Luis
On 06/13/2013 07:10 PM, pe...@2force.nl wrote: On 2013-06-13 18:57, Joao Eduardo Luis wrote: On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 13, 2013, wrote: Hello, We ran into a problem with our test cluster after adding monitors.

Re: [ceph-users] Disaster recovery of monitor

2013-06-14 Thread peter
On 2013-06-13 20:10, pe...@2force.nl wrote: On 2013-06-13 18:57, Joao Eduardo Luis wrote: On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 13, 2013, wrote: Hello, We ran into a problem with our test cluster after adding monitors. It

Re: [ceph-users] Disaster recovery of monitor

2013-06-13 Thread peter
On 2013-06-13 18:57, Joao Eduardo Luis wrote: On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 13, 2013, wrote: Hello, We ran into a problem with our test cluster after adding monitors. It now seems that our main monitor doesn't wan

Re: [ceph-users] Disaster recovery of monitor

2013-06-13 Thread Joao Eduardo Luis
On 06/13/2013 05:25 PM, pe...@2force.nl wrote: On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 13, 2013, wrote: Hello, We ran into a problem with our test cluster after adding monitors. It now seems that our main monitor doesn't want to start anymore. The logs are flooded with: 20

Re: [ceph-users] Disaster recovery of monitor

2013-06-13 Thread peter
On 2013-06-13 18:06, Gregory Farnum wrote: On Thursday, June 13, 2013, wrote: Hello, We ran into a problem with our test cluster after adding monitors. It now seems that our main monitor doesn't want to start anymore. The logs are flooded with: 2013-06-13 11:41:05.316982 7f7689ca4780  7 mon.a

Re: [ceph-users] Disaster recovery of monitor

2013-06-13 Thread Gregory Farnum
On Thursday, June 13, 2013, wrote: > Hello, > > We ran into a problem with our test cluster after adding monitors. It now > seems that our main monitor doesn't want to start anymore. The logs are > flooded with: > > 2013-06-13 11:41:05.316982 7f7689ca4780 7 mon.a@0(leader).osd e2809 > update_from

[ceph-users] Disaster recovery of monitor

2013-06-13 Thread peter
Hello, We ran into a problem with our test cluster after adding monitors. It now seems that our main monitor doesn't want to start anymore. The logs are flooded with: 2013-06-13 11:41:05.316982 7f7689ca4780 7 mon.a@0(leader).osd e2809 update_from_paxos applying incremental 2810 2013-06-13

[ceph-users] Disaster recovery of monitor

2013-06-13 Thread peter
Hello, We ran into a problem with our test cluster after adding monitors. It now seems that our main monitor doesn't want to start anymore. The logs are flooded with: 2013-06-13 11:41:05.316982 7f7689ca4780 7 mon.a@0(leader).osd e2809 update_from_paxos applying incremental 2810 2013-06-13