On Tue, Oct 4, 2016 at 4:27 PM, Stephen Horton <short...@yahoo.com> wrote: > Adding that all of my ceph components are version: > 10.2.2-0ubuntu0.16.04.2 > > Openstack is Mitaka on Ubuntu 16.04x. Manila file share is 1:2.0.0-0ubuntu1 > > My scenario is that I have a 3-node ceph cluster running openstack mitaka. > Each node has 256gb ram, 14tb raid 5 array. I have 30 VMs running in > openstack; all are mounted to the Manila file share using cephfs native > kernel client driver. Each VM user has put 10-20 gb of files on the share, > but most of this is back-up, so IO requirement is very low. However, I > initially tried using ceph-fuse but performance latency was poor. Moving to > kernel client driver for mounting the share has improved performance greatly. > However, I am getting the cache pressure issue.
Aside: bear in mind that the kernel client doesn't support quotas, so any size limits you set on your Manila shares won't be respected. > Can someone help me with the math to properly size the mds cache? How do I > know if the cache size is too small (I think very few files in-use at any > given time) versus the clients are broken and not releasing cache properly? It's almost never the case that your cache is too small unless your workload is holding a silly number of files open at one time -- assume this is a client bug (although some people work around it by creating much bigger MDS caches!) You've mentioned the versions of openstack/ubuntu/ceph, but what kernel are you running? John > Thank you! > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
