Re: [ceph-users] redundancy with 2 nodes

2014-12-31 Thread Mark Kirkwood
The number of monitors recommended and the fact that a voting quorum is the way it works is covered here: http://ceph.com/docs/master/rados/deployment/ceph-deploy-mon/ but I agree that you should probably not get a HEALTH OK status when you have just setup 2 (or in fact any even number of) mon

Re: [ceph-users] redundancy with 2 nodes

2014-12-31 Thread Jiri Kanicky
Hi, I think you are right. I was too focused on the following line in docs: "A cluster will run fine with a single monitor; however,*a single monitor is a single-point-of-failure*." I will try to add another monitor. Hopefully, this will fix my issue. Anyway, I think that "ceph status" or "c

Re: [ceph-users] redundancy with 2 nodes

2014-12-31 Thread Christian Balzer
On Thu, 01 Jan 2015 14:59:05 +1100 Jiri Kanicky wrote: > > monmap e1: 2 mons at > {ceph1=192.168.30.21:6789/0,ceph2=192.168.30.22:6789/0}, election epoch > 12, quorum 0,1 ceph1,ceph2 > That's your problem, re-read the Ceph documentation about Paxos. You need a third monitor to retain a vi

Re: [ceph-users] redundancy with 2 nodes

2014-12-31 Thread Lindsay Mathieson
On Thu, 1 Jan 2015 03:46:33 PM Jiri Kanicky wrote: > Hi, > > I have: > - 2 monitors, one on each node > - 4 OSDs, two on each node > - 2 MDS, one on each node POOMA U here, but I don't think you can reach quorum with one out of two monitors, you need a odd number: http://ceph.com/docs/master/ra

Re: [ceph-users] redundancy with 2 nodes

2014-12-31 Thread Jiri Kanicky
Hi, I have: - 2 monitors, one on each node - 4 OSDs, two on each node - 2 MDS, one on each node Yes, all pools are set with size=2 and min_size=1 cephadmin@ceph1:~$ ceph osd dump epoch 88 fsid bce2ff4d-e03b-4b75-9b17-8a48ee4d7788 created 2014-12-27 23:38:00.455097 modified 2014-12-30 20:45:51.3

Re: [ceph-users] redundancy with 2 nodes

2014-12-31 Thread Lindsay Mathieson
On Thu, 1 Jan 2015 02:59:05 PM Jiri Kanicky wrote: > I would expect that if I shut down one node, the system will keep > running. But when I tested it, I cannot even execute "ceph status" > command on the running node. 2 osd Nodes, 3 Mon nodes here, works perfectly for me. How many monitors do

[ceph-users] redundancy with 2 nodes

2014-12-31 Thread Jiri Kanicky
Hi, Is it possible to achieve redundancy with 2 nodes only? cephadmin@ceph1:~$ ceph osd tree # idweight type name up/down reweight -1 10.88 root default -2 5.44host ceph1 0 2.72osd.0 up 1 1 2.72osd.1

[ceph-users] Weighting question

2014-12-31 Thread Lindsay Mathieson
As mentioned before :) we have two osd ndoes with one 3TB osd each. (replica 2) About to add a smaller (1TB) faster drive to each node From the docs, normal practice would be to weight it in accordance with size, i.e 3 for the 3TB OSD, 1 for the 1TB OSD. But I'd like to spread it 50/50 to take

Re: [ceph-users] Marking a OSD a new in the OSDMap

2014-12-31 Thread Andrey Korolyov
On Wed, Dec 31, 2014 at 8:20 PM, Wido den Hollander wrote: > On 12/31/2014 05:54 PM, Andrey Korolyov wrote: >> On Wed, Dec 31, 2014 at 7:34 PM, Wido den Hollander wrote: >>> Hi, >>> >>> Is there a way to set a OSD to exists,new in the OSDMap? I want to >>> re-install a OSD and re-use the ID and i

Re: [ceph-users] Marking a OSD a new in the OSDMap

2014-12-31 Thread Wido den Hollander
On 12/31/2014 05:54 PM, Andrey Korolyov wrote: > On Wed, Dec 31, 2014 at 7:34 PM, Wido den Hollander wrote: >> Hi, >> >> Is there a way to set a OSD to exists,new in the OSDMap? I want to >> re-install a OSD and re-use the ID and it's key. >> >> I know I can remove the OSD and re-add it, but that

Re: [ceph-users] Marking a OSD a new in the OSDMap

2014-12-31 Thread Andrey Korolyov
On Wed, Dec 31, 2014 at 7:34 PM, Wido den Hollander wrote: > Hi, > > Is there a way to set a OSD to exists,new in the OSDMap? I want to > re-install a OSD and re-use the ID and it's key. > > I know I can remove the OSD and re-add it, but that triggers balancing > and I want to prevent that by simp

Re: [ceph-users] Marking a OSD a new in the OSDMap

2014-12-31 Thread Luis Periquito
Hi Wido, FWIW the last time I had a failed disk I just removed the old OSD and created a new one with the same ID, and it only rebalanced the "new" OSD. But I'll admit that I didn't pay much attention to the process. If you just want to reformat the disk you have to be carefull with the Journal.

[ceph-users] Marking a OSD a new in the OSDMap

2014-12-31 Thread Wido den Hollander
Hi, Is there a way to set a OSD to exists,new in the OSDMap? I want to re-install a OSD and re-use the ID and it's key. I know I can remove the OSD and re-add it, but that triggers balancing and I want to prevent that by simply marking the OSD a new and booting it with a freshly formatted XFS fil

Re: [ceph-users] Crush Map and SSD Pools

2014-12-31 Thread Damien Churchill
On Tue Dec 30 2014 at 11:49:09 PM Lindsay Mathieson < lindsay.mathie...@gmail.com> wrote: > On Tue, 30 Dec 2014 11:25:40 PM Erik Logtenberg wrote: > > f you want to be able to start your osd's with /etc/init.d/ceph init > > script, then you better make sure that /etc/ceph/ceph.conf does link > > t

Re: [ceph-users] Crush Map and SSD Pools

2014-12-31 Thread Lindsay Mathieson
On Wed, 31 Dec 2014 11:09:35 AM you wrote: > I believe that the upstart scripts will do this by default, they call out to > a bash script (I can't remember precisely what that is off the top of my > head) which then returns the crush rule, which will default to host=X osd=X > unless it's overridden

[ceph-users] radosgw RX much more than TX

2014-12-31 Thread Mustafa Muhammad
Hi, I am having a problem with radosgw, RX is much more than TX, sometimes more than 10x. This happens with apache and with civetweb. I couldn't find anything wrong in the logs (only "RGWObjManifest::operator++"), what should I do? I didn't know where to ask (here or dev) Regards Mustafa Muhammad