We have graphs for network usage in graphana. We even have aggregate graphs for projects. For my team, we specifically have graphs for the Ceph cluster osd public network, osd private network, rgw network, and mon network. You should be able to do something similar for each of the servers in your project to know how much network traffic they are using, especially if they were using a vlan to access the ceph cluster where nothing else used that interface on the server. It would only report cephfs traffic on each client node.
I agree that pool per project is a poor design when you don't have a finite and limited number of projects. On Fri, Dec 8, 2017 at 9:44 AM Martin Dojcak <doj...@firma.zoznam.sk> wrote: > Hi > > we are planning to replace our NFS infra with CephFS (Luminous). Our use > case for CephFS is mounting directories via kernel client (not fuse for > performance reason). The cephfs root directory is logically split into > subdirs where each represent separate project with its own source codes > and application data, for example: > . > ├── project-1 > ├── project-2 > ├── project-3 > ├── project-4 > ├── project-5 > └── project-N > > Project could be mounted only with client that has right permissions. > There are few options to do that: > 1. all projects are in default data pool with default namespace and > client got permissions via mds cap path restriction (ceph auth > get-or-create client.project-1 mds 'allow rw path=/project-1' ...) > 2. all projects are in default data pool with custom namespace per > project and client got permissions via osd cap namespace restriction > (ceph auth get-or-create client.project-1 namespace=project-1 ...) > 3. each project is in separate data pool and client got permissions via > osd cap pool restriction (ceph auth get-or-create client.project-1 > pool=project-1) > > One `must have` for us is possibility to monitor per project I/O > activity and throughput. Sometimes you need to know what project > generate high load on storage and fix it. > With first and second options there is no way to accomplish that. I > search around for tool like iostat for cephfs kernel client but for now > it doesnt exists. > Third option can be monitored by `ceph osd pool stats` command but there > are some disadvantages when you have many pools: > - recovery and backfilling problems > - problem to stay with predefined PGs per OSD > - problem to have good number of PGs per pool > - many more > > Any ideas how can we monitor load per project on cephfs or per client ? > > Martin Dojcak > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
