Thanks John. I'll give that a try as soon as I fix an issue with my MDS servers that cropped up today.
On Mon, Aug 10, 2015 at 2:58 AM, John Spray <jsp...@redhat.com> wrote: > On Fri, Aug 7, 2015 at 1:36 AM, Bob Ababurko <b...@ababurko.net> wrote: > > @John, > > > > Can you clarify which values would suggest that my metadata pool is too > > slow? I have added a link that includes values for the "op_active" & > > "handle_client_request"....gathered in a crude fashion but should > hopefully > > give enough data to paint a picture of what is happening. > > > > http://pastebin.com/5zAG8VXT > > Dividing by the first 20s of the second sample period, you're seeing > ~750 client metadata operations handled per second, which is kind of a > baseline level of performance (a little better than what I get running > a ceph cluster locally on my workstation). That's probably > corresponding to roughly the same number of file creates per second -- > your workload is very much a small file one, where "files per second" > is a much more meaningful measure than IOPS or MB/s. > > It does look like the kind of pattern where you've got a large clutch > of several thousand metadata pool rados ops coming out every few > seconds, then draining out over a few seconds. Your metadata pool > isn't pathologically slow (it's completing at least hundreds of ops > per second), but it is noticeable that during some periods where > op_active is draining, handle_client_request is not incrementing -- > i.e. client metadata ops are stalling while the MDS waits for its > RADOS operations to complete. > > I can't say a massive amount beyond that, other than what you'd > already figured out -- it would be worth trying to put some faster > storage in for your metadata pool. > > John >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
