i think you may increase mds_bal_fragment_size_max, default is 100000
> > On Oct 4, 2016, at 10:30 AM, John Spray <jsp...@redhat.com> wrote: > > > >> On Tue, Oct 4, 2016 at 5:09 PM, Stephen Horton <short...@yahoo.com> > wrote: > >> Thank you John. Both my Openstack hosts and the VMs are all running > 4.4.0-38-generic #57-Ubuntu SMP x86_64. I can see no evidence that any of > the VMs are holding large numbers of files open. If this is likely a client > bug, is there some process I can follow to file a bug report? > > > > It might be worthwhile to file a bug report with Ubuntu, as they'd be > > the ones who would ideally backport fixes to their stable kernels (in > > this instance it's hard to know if this is a bug in the latest kernel > > code or something fixed since 4.4). > > > > It would be really useful if you could try installing the latest > > released kernel on the clients and see if the issue persists: if so > > then a ticket on tracker.ceph.com will be a priority for us to fix. > > > > CCing Zheng -- are there any noteworthy fixes between 4.4 and latest > > kernel that might be relevant? > > > > John > > > > > >> > >>>> On Oct 4, 2016, at 9:39 AM, John Spray <jsp...@redhat.com> wrote: > >>>> > >>>> On Tue, Oct 4, 2016 at 4:27 PM, Stephen Horton <short...@yahoo.com> > wrote: > >>>> Adding that all of my ceph components are version: > >>>> 10.2.2-0ubuntu0.16.04.2 > >>>> > >>>> Openstack is Mitaka on Ubuntu 16.04x. Manila file share is > 1:2.0.0-0ubuntu1 > >>>> > >>>> My scenario is that I have a 3-node ceph cluster running openstack > mitaka. Each node has 256gb ram, 14tb raid 5 array. I have 30 VMs running > in openstack; all are mounted to the Manila file share using cephfs native > kernel client driver. Each VM user has put 10-20 gb of files on the share, > but most of this is back-up, so IO requirement is very low. However, I > initially tried using ceph-fuse but performance latency was poor. Moving to > kernel client driver for mounting the share has improved performance > greatly. However, I am getting the cache pressure issue. > >>> > >>> Aside: bear in mind that the kernel client doesn't support quotas, so > >>> any size limits you set on your Manila shares won't be respected. > >>> > >>>> Can someone help me with the math to properly size the mds cache? How > do I know if the cache size is too small (I think very few files in-use at > any given time) versus the clients are broken and not releasing cache > properly? > >>> > >>> It's almost never the case that your cache is too small unless your > >>> workload is holding a silly number of files open at one time -- assume > >>> this is a client bug (although some people work around it by creating > >>> much bigger MDS caches!) > >>> > >>> You've mentioned the versions of openstack/ubuntu/ceph, but what > >>> kernel are you running? > >>> > >>> John > >>> > >>>> Thank you! > >>>> _______________________________________________ > >>>> ceph-users mailing list > >>>> ceph-users@lists.ceph.com > >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >> >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
