Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Adrian Saul
automount would probably work for you. From: Up Safe [mailto:upands...@gmail.com] Sent: Tuesday, 22 May 2018 12:33 AM To: David Turner Cc: Adrian Saul ; ceph-users Subject: Re: [ceph-users] multi site with cephfs I'll explain. Right now we have 2 sites (racks) with several dozens of servers at

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Janne Johansson
Den mån 21 maj 2018 kl 18:28 skrev Up Safe : > I don't believe I have this kind of behavior. > AFAIK, files are created or modified by only 1 client at a time. > Make sure that this is the case then, its _very_ easy to start out with something along the lines of "right now I cannot image this" an

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Up Safe
for our usage but is noticeably >>>>>> slower >>>>>> than comparable local ceph based RBD shares. >>>>>> >>>>>> >>>>>> >>>>>> Essentially we just setup the ceph pools behind cephFS to ha

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Brady Deetz
cephFS to have >>>>> replicas on each site. To export it we are simply using Linux kernel NFS >>>>> and it gets exported from 4 hosts that act as CephFS clients. Those 4 >>>>> hosts are then setup in an DNS record that resolves to all 4 IPs, and

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Brady Deetz
d cluster of about >>>>>>> 40km with redundant 10G fibre between sites – link latency is in the >>>>>>> order >>>>>>> of 1-2ms. Performance is reasonable for our usage but is noticeably >>>>>>> slower >>>>>&

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Up Safe
mparable local ceph based RBD shares. >>>>>> >>>>>> >>>>>> >>>>>> Essentially we just setup the ceph pools behind cephFS to have >>>>>> replicas on each site. To export it we are simply using Linux kernel NFS >

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Paul Emmerich
;> and it gets exported from 4 hosts that act as CephFS clients. Those 4 >>>>> hosts are then setup in an DNS record that resolves to all 4 IPs, and we >>>>> then use automount to do automatic mounting and host failover on the NFS >>>>> clients. Autom

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Up Safe
and it >>>> gets exported from 4 hosts that act as CephFS clients. Those 4 hosts are >>>> then setup in an DNS record that resolves to all 4 IPs, and we then use >>>> automount to do automatic mounting and host failover on the NFS clients. >>>> Automo

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread David Turner
>>> >>> >>> >>> I stress this is a limited setup that we use for some fairly light duty, >>> but we are looking to move things like user home directories onto this. >>> YMMV. >>> >>> >>> >>> >>>

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Up Safe
oking to move things like user home directories onto this. >> YMMV. >> >> >> >> >> >> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf >> Of *Up Safe >> *Sent:* Monday, 21 May 2018 5:36 PM >> *To:* David Turner &

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Up Safe
is a limited setup that we use for some fairly light duty, > but we are looking to move things like user home directories onto this. > YMMV. > > > > > > *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf > Of *Up Safe > *Sent:* Monday, 21 May 2018 5:3

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Adrian Saul
-users-boun...@lists.ceph.com] On Behalf Of Up Safe Sent: Monday, 21 May 2018 5:36 PM To: David Turner Cc: ceph-users Subject: Re: [ceph-users] multi site with cephfs Hi, can you be a bit more specific? I need to understand whether this is doable at all. Other options would be using ganesha, but I

Re: [ceph-users] multi site with cephfs

2018-05-21 Thread Up Safe
Hi, can you be a bit more specific? I need to understand whether this is doable at all. Other options would be using ganesha, but I understand it's very limited on NFS; or start looking at gluster. Basically, I need the multi site option, i.e. active-active read-write. Thanks On Wed, May 16, 20

Re: [ceph-users] multi site with cephfs

2018-05-16 Thread David Turner
Object storage multi-site is very specific to using object storage. It uses the RGW API's to sync s3 uploads between each site. For CephFS you might be able to do a sync of the rados pools, but I don't think that's actually a thing yet. RBD mirror is also a layer on top of things to sync between

Re: [ceph-users] multi site with cephfs

2018-05-16 Thread Up Safe
But this is not the question here. The question is whether I can configure multi site for CephFS. Will I be able to do so by following the guide to set up the multi site for object storage? Thanks On Wed, May 16, 2018, 16:45 John Hearns wrote: > The answer given at the seminar yesterday was tha

Re: [ceph-users] multi site with cephfs

2018-05-16 Thread John Hearns
The answer given at the seminar yesterday was that a practical limit was around 60km. I don't think 100km is that much longer. I defer to the experts here. On 16 May 2018 at 15:24, Up Safe wrote: > Hi, > > About a 100 km. > I have a 2-4ms latency between them. > > Leon > > On Wed, May 16,

Re: [ceph-users] multi site with cephfs

2018-05-16 Thread Up Safe
Hi, About a 100 km. I have a 2-4ms latency between them. Leon On Wed, May 16, 2018, 16:13 John Hearns wrote: > Leon, > I was at a Lenovo/SuSE seminar yesterday and asked a similar question > regarding separated sites. > How far apart are these two geographical locations? It does matter. > >

Re: [ceph-users] multi site with cephfs

2018-05-16 Thread John Hearns
Leon, I was at a Lenovo/SuSE seminar yesterday and asked a similar question regarding separated sites. How far apart are these two geographical locations? It does matter. On 16 May 2018 at 15:07, Up Safe wrote: > Hi, > > I'm trying to build a multi site setup. > But the only guides I've found