Hi Jeff

Thanks for the pointer on this.  I found some details on this the other day and 
your link is a big help.  I will get this updated in my ansible playbook and 
test.

Rgds

Lee

On 01/08/2019 17:03, Jeff Layton wrote:

On Sun, 2019-07-28 at 18:20 +0000, Lee Norvall wrote:


Update to this I found that you cannot create a 2nd files system as yet and it 
is still experimental.  So I went down this route:

Added a pool to the existing cephfs and then setfattr -n ceph.dir.layout.pool 
-v SSD-NFS /mnt/cephfs/ssdnfs/ from a ceph-fuse client.

I then nfs mounted from another box. I can see the files and dir etc from the 
nfs client but my issue now is that I do not have permission to write, create 
dir etc.  The same goes for the default setup after running the ansible 
playbook even when setting export to no_root_squash.  I am missing a chain of 
permission?  ganesha-nfs is using admin userid, is this the same as the 
client.admin or is this a user I need to create?  Any info appreciated.

Ceph is on CentOS 7 and SELinux is currently off as well.

Copy of the ganesha conf below.  Is secType correct or is it missing something?

RADOS_URLS {
   ceph_conf = '/etc/ceph/ceph.conf';
   userid = "admin";
}
%url rados://cephfs_data/ganesha-export-index

NFSv4 {
        RecoveryBackend = 'rados_kv';
}



I your earlier email, you mentioned that you had more than one NFS
server, but rados_kv is not safe in a multi-server configuration. The
servers will be competing to store recovery information in the same
objects, and won't honor each others' grace periods/

You may want to explore using "RecoveryBackend = rados_cluster" instead,
which should handle that situation better. See this writeup, for some
guidelines:

    
https://jtlayton.wordpress.com/2018/12/10/deploying-an-active-active-nfs-cluster-over-cephfs/

Much of this is already automated too if you use k8s+rook.



RADOS_KV {
        ceph_conf = '/etc/ceph/ceph.conf';
        userid = "admin";
        pool = "cephfs_data";
}

EXPORT
{
        Export_id=20133;
        Path = "/";
        Pseudo = /cephfile;
        Access_Type = RW;
        Protocols = 3,4;
        Transports = TCP;
        SecType = sys,krb5,krb5i,krb5p;
        Squash = Root_Squash;
        Attr_Expiration_Time = 0;

        FSAL {
                Name = CEPH;
                User_Id = "admin";
        }


}










On 28/07/2019 12:11, Lee Norvall wrote:


Hi

I am using ceph-ansible to deploy and just looking for best way/tips on
how to export multiple pools/fs.

Ceph: nautilus (14.2.2)
NFS-Ganesha v 2.8
ceph-ansible stable 4.0

I have 3 x osd/NFS gateways running and NFS on the dashboard can see
them in the cluster.  I have managed to export for cephfs / and mounted
it on another box.

1) can I add a new pool/fs to the export under that same NFS gateway
cluster, or

2) do I have the to do something like add a new pool to the fs and then
setfattr to make the layout /newfs_dir point to /new_pool?  does this
cause issues and false object count?

3) any other better ways...

Rgds

Lee

_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



--


Lee Norvall | CEO / Founder
Mob. +44 (0)7768 201884
Tel. +44 (0)20 3026 8930
Web. www.blocz.io<http://www.blocz.io>

Enterprise Cloud | Private Cloud | Hybrid/Multi Cloud | Cloud Backup



This e-mail (and any attachment) has been sent from a PC belonging to My Mind 
(Holdings) Limited. If you receive it in error, please tell us by return and 
then delete it from your system; you may not rely on its contents nor 
copy/disclose it to anyone. Opinions, conclusions and statements of intent in 
this e-mail are those of the sender and will not bind My Mind (Holdings) 
Limited unless confirmed by an authorised representative independently of this 
message. We do not accept responsibility for viruses; you must scan for these. 
Please note that e-mails sent to and from blocz IO Limited are routinely 
monitored for record keeping, quality control and training purposes, to ensure 
regulatory compliance and to prevent viruses and unauthorised use of our 
computer systems. My Mind (Holdings) Limited is registered in England & Wales 
under company number 10186410. Registered office: 1st Floor Offices, 2a 
Highfield Road, Ringwood, Hampshire, United Kingdom, BH24 1RQ. VAT Registration 
GB 244 9628 77
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com





--
[blocz IO Limted]

Lee Norvall | CEO / Founder
Mob. +44 (0)7768 201884
Tel. +44 (0)20 3026 8930
Web. www.blocz.io<http://www.blocz.io>

Enterprise Cloud | Private Cloud | Hybrid/Multi Cloud | Cloud Backup

[blocz IO Limted]   [blocz IO Limted]    [blocz IO Limted]    [blocz IO Limted] 
   [blocz IO Limted]    [ESET MSP Partner]

This e-mail (and any attachment) has been sent from a PC belonging to My Mind 
(Holdings) Limited. If you receive it in error, please tell us by return and 
then delete it from your system; you may not rely on its contents nor 
copy/disclose it to anyone. Opinions, conclusions and statements of intent in 
this e-mail are those of the sender and will not bind My Mind (Holdings) 
Limited unless confirmed by an authorised representative independently of this 
message. We do not accept responsibility for viruses; you must scan for these. 
Please note that e-mails sent to and from blocz IO Limited are routinely 
monitored for record keeping, quality control and training purposes, to ensure 
regulatory compliance and to prevent viruses and unauthorised use of our 
computer systems. My Mind (Holdings) Limited is registered in England & Wales 
under company number 10186410. Registered office: 1st Floor Offices, 2a 
Highfield Road, Ringwood, Hampshire, United Kingdom, BH24 1RQ. VAT Registration 
GB 244 9628 77
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to