Using CephFS for something like this is about the last thing I would do. Does it need to be on a networked posix filesystem that can be mounted on multiple machines at the same time? If so, then you're kinda stuck and we can start looking at your MDS hardware and see if there are any MDS settings that need to be configured differently for this to work.
If you don't NEED CephFS, then I would recommend utilizing an RBD for something like this. Its limitation is only being able to be mapped to 1 server at a time, but that's decent enough for most failover scenarios for build setups. If you need to failover, unmap it from the primary and map it to another server to resume workloads. Hosting websites out of CephFS also seems counter-intuitive. Have you looked at S3 websites? RGW supports configuring websites out of a bucket that might be of interest. Your RGW daemon configuration could easily become an HA website with an LB in front of them. I'm biased here a bit, but I don't like to use networked filesystems unless nothing else can be worked out or the software using it is 3rd party and just doesn't support anything else. On Thu, Mar 1, 2018 at 9:05 AM Daniel Carrasco <d.carra...@i2tic.com> wrote: > Hello, > > I've tried to change a lot of things on configuration and use ceph-fuse > but nothing makes it work better... When I deploy the git repository it > becomes much slower until I remount the FS (just executing systemctl stop > nginx && umount /mnt/ceph && mount -a && systemctl start nginx). It happen > when the FS gets a lot of IO because when I execute Rsync I got the same > problem. > > I'm thinking about to downgrade to a lower version of ceph like for > example jewel to see if works better. I know that will be deprecated soon, > but I don't know what other tests I can do... > > Greetings!! > > 2018-02-28 17:11 GMT+01:00 Daniel Carrasco <d.carra...@i2tic.com>: > >> Hello, >> >> I've created a Ceph cluster with 3 nodes and a FS to serve a webpage. The >> webpage speed is good enough (near to NFS speed), and have HA if one FS die. >> My problem comes when I deploy a git repository on that FS. The server >> makes a lot of IOPS to check the files that have to update and then all >> clients starts to have problems to use the FS (it becomes much slower). >> In a normal usage the web takes about 400ms to load, and when the problem >> start it takes more than 3s. To fix the problem I just have to remount the >> FS on clients, but I can't remount the FS on every deploy... >> >> While is deploying I see how the CPU on MDS is a bit higher, but when it >> ends the CPU usage goes down again, so look like is not a problem of CPU. >> >> My config file is: >> [global] >> fsid = bf56854......e611c08 >> mon_initial_members = fs-01, fs-02, fs-03 >> mon_host = 10.50.0.94,10.50.1.216,10.50.2.52 >> auth_cluster_required = cephx >> auth_service_required = cephx >> auth_client_required = cephx >> >> public network = 10.50.0.0/22 >> osd pool default size = 3 >> >> ## >> ### OSD >> ## >> [osd] >> osd_mon_heartbeat_interval = 5 >> osd_mon_report_interval_max = 10 >> osd_heartbeat_grace = 15 >> osd_fast_fail_on_connection_refused = True >> osd_pool_default_pg_num = 128 >> osd_pool_default_pgp_num = 128 >> osd_pool_default_size = 2 >> osd_pool_default_min_size = 2 >> >> ## >> ### Monitores >> ## >> [mon] >> mon_osd_min_down_reporters = 1 >> >> ## >> ### MDS >> ## >> [mds] >> mds_cache_memory_limit = 792723456 >> mds_bal_mode = 1 >> >> ## >> ### Client >> ## >> [client] >> client_cache_size = 32768 >> client_mount_timeout = 30 >> client_oc_max_objects = 2000 >> client_oc_size = 629145600 >> client_permissions = false >> rbd_cache = true >> rbd_cache_size = 671088640 >> >> My cluster and clients uses Debian 9 with latest ceph version (12.2.4). >> The clients uses kernel modules to mount the share, because are a bit >> faster than fuse modules. The deploy is done on one of the Ceph nodes, that >> have the FS mounted by kernel module too. >> My cluster is not a high usage cluster, so have all daemons on one >> machine (3 machines with OSD, MON, MGR and MDS). All OSD has a copy of the >> data, only one MGR is active and two of the MDS are active with one on >> standby. The clients mount the FS using the three MDS IP addresses and just >> now don't have any request because is not published. >> >> Someone knows what can be happening?, because all works fine (even on >> other cluster I did with an high load), but just deploy the git repository >> and all start to work very slow. >> >> Thanks!! >> >> >> -- >> _________________________________________ >> >> Daniel Carrasco Marín >> Ingeniería para la Innovación i2TIC, S.L. >> Tlf: +34 911 12 32 84 Ext: 223 >> www.i2tic.com >> _________________________________________ >> > > > > -- > _________________________________________ > > Daniel Carrasco Marín > Ingeniería para la Innovación i2TIC, S.L. > Tlf: +34 911 12 32 84 Ext: 223 > www.i2tic.com > _________________________________________ > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
