Re: [ceph-users] Location of MONs

2013-07-23 Thread Alex Bligh
On 23 Jul 2013, at 17:16, Gregory Farnum wrote: >> And without wanting to sound daft having missed a salient configuration >> detail, but there's no way to release when it's written the primary? > > Definitely not. Ceph's consistency guarantees and recovery mechanisms > are all built on top of a

Re: [ceph-users] Location of MONs

2013-07-23 Thread Matthew Walster
On 23 July 2013 17:16, Gregory Farnum wrote: > > And without wanting to sound daft having missed a salient configuration > > detail, but there's no way to release when it's written the primary? > > Definitely not. > ​I thought as much -- and now you've explained why, I completely understand the

Re: [ceph-users] Location of MONs

2013-07-23 Thread Gregory Farnum
On Tue, Jul 23, 2013 at 9:12 AM, Matthew Walster wrote: > On 23 July 2013 17:07, Gregory Farnum wrote: >> >> If you have three osds that are >> separated by 5ms each and all hosting a PG, then your lower-bound >> latency for a write op is 10ms — 5 ms to send from the primary to the >> replicas,

Re: [ceph-users] Location of MONs

2013-07-23 Thread Matthew Walster
On 23 July 2013 17:07, Gregory Farnum wrote: > If you have three osds that are > separated by 5ms each and all hosting a PG, then your lower-bound > latency for a write op is 10ms — 5 ms to send from the primary to the > replicas, 5ms for them to ack back. And without wanting to sound daft ha

Re: [ceph-users] Location of MONs

2013-07-23 Thread Gregory Farnum
On Tue, Jul 23, 2013 at 9:04 AM, Matthew Walster wrote: > > That's fantastic, thanks. I'm assuming that 5ms is probably too much for the > OSDs -- do we have any idea/data as to the effect of latency on OSDs if they > were split over a similar distance? Or even a spread - 0.5ms, 1ms, 2ms etc. > Ob

Re: [ceph-users] Location of MONs

2013-07-23 Thread Matthew Walster
On 23 July 2013 16:59, Gregory Farnum wrote: > On Tue, Jul 23, 2013 at 8:54 AM, Matthew Walster > wrote: > > I've got a relatively small Ceph cluster I'm playing with at the moment, > and > > against advice, I'm running the MONs on the OSDs. > > > > Ideally, I'd like to have half the OSDs in a d

Re: [ceph-users] Location of MONs

2013-07-23 Thread Gregory Farnum
On Tue, Jul 23, 2013 at 8:54 AM, Matthew Walster wrote: > I've got a relatively small Ceph cluster I'm playing with at the moment, and > against advice, I'm running the MONs on the OSDs. > > Ideally, I'd like to have half the OSDs in a different facility and > therefore have one MON in each facili

[ceph-users] Location of MONs

2013-07-23 Thread Matthew Walster
I've got a relatively small Ceph cluster I'm playing with at the moment, and against advice, I'm running the MONs on the OSDs. Ideally, I'd like to have half the OSDs in a different facility and therefore have one MON in each facility. Which gives the problem of where to put that pesky third MON.