Re: [ceph-users] CephFS and caching

2015-09-14 Thread Gregory Farnum
On Thu, Sep 10, 2015 at 1:07 PM, Kyle Hutson wrote: > A 'rados -p cachepool ls' takes about 3 hours - not exactly useful. > > I'm intrigued that you say a single read may not promote it into the cache. > My understanding is that if you have an EC-backed pool the clients can't > talk to them direct

Re: [ceph-users] CephFS and caching

2015-09-11 Thread Ilya Dryomov
On Wed, Sep 9, 2015 at 5:34 PM, Gregory Farnum wrote: > On Wed, Sep 9, 2015 at 3:27 PM, Kyle Hutson wrote: >> We are using Hammer - latest released version. How do I check if it's >> getting promoted into the cache? > > Umm...that's a good question. You can run rados ls on the cache pool, > but t

Re: [ceph-users] CephFS and caching

2015-09-10 Thread Kyle Hutson
A 'rados -p cachepool ls' takes about 3 hours - not exactly useful. I'm intrigued that you say a single read may not promote it into the cache. My understanding is that if you have an EC-backed pool the clients can't talk to them directly, which means they would necessarily be promoted to the cach

Re: [ceph-users] CephFS and caching

2015-09-09 Thread Gregory Farnum
On Wed, Sep 9, 2015 at 4:26 PM, Kyle Hutson wrote: > > > On Wed, Sep 9, 2015 at 9:34 AM, Gregory Farnum wrote: >> >> On Wed, Sep 9, 2015 at 3:27 PM, Kyle Hutson wrote: >> > We are using Hammer - latest released version. How do I check if it's >> > getting promoted into the cache? >> >> Umm...tha

Re: [ceph-users] CephFS and caching

2015-09-09 Thread Kyle Hutson
On Wed, Sep 9, 2015 at 9:34 AM, Gregory Farnum wrote: > On Wed, Sep 9, 2015 at 3:27 PM, Kyle Hutson wrote: > > We are using Hammer - latest released version. How do I check if it's > > getting promoted into the cache? > > Umm...that's a good question. You can run rados ls on the cache pool, > bu

Re: [ceph-users] CephFS and caching

2015-09-09 Thread Gregory Farnum
On Wed, Sep 9, 2015 at 3:27 PM, Kyle Hutson wrote: > We are using Hammer - latest released version. How do I check if it's > getting promoted into the cache? Umm...that's a good question. You can run rados ls on the cache pool, but that's not exactly scalable; you can turn up logging and dig into

Re: [ceph-users] CephFS and caching

2015-09-09 Thread Kyle Hutson
We are using Hammer - latest released version. How do I check if it's getting promoted into the cache? We're using the latest ceph kernel client. Where do I poke at readahead settings there? On Tue, Sep 8, 2015 at 8:29 AM, Gregory Farnum wrote: > On Thu, Sep 3, 2015 at 11:58 PM, Kyle Hutson wr

Re: [ceph-users] CephFS and caching

2015-09-08 Thread Gregory Farnum
On Thu, Sep 3, 2015 at 11:58 PM, Kyle Hutson wrote: > I was wondering if anybody could give me some insight as to how CephFS does > its caching - read-caching in particular. > > We are using CephFS with an EC pool on the backend with a replicated cache > pool in front of it. We're seeing some very

Re: [ceph-users] CephFS and caching

2015-09-04 Thread Les
Cephfs can use fscache. I am testing it at the moment. Some lines from my deployment process: sudo apt-get install linux-generic-lts-utopic cachefilesd sudo reboot sudo mkdir /mnt/cephfs sudo mkdir /mnt/ceph_cache sudo mkfs -t xfs /dev/md3 # A 100gb local raid partition sudo bash -c "echo /dev/md

[ceph-users] CephFS and caching

2015-09-03 Thread Kyle Hutson
I was wondering if anybody could give me some insight as to how CephFS does its caching - read-caching in particular. We are using CephFS with an EC pool on the backend with a replicated cache pool in front of it. We're seeing some very slow read times. Trying to compute an md5sum on a 15GB file t