This is a known issue. I've had this issue pop up after several days for no apparent reason. At this point I'm thinking the keyring will need to be configurable to provide a workaround. Ideally the keys would be added to a Lustre specific keyring so everything would be cleaned up when the modules are removed However, the problem there is they need to be loaded before the mount system call.
Please file a bug if you have a chance. Jeremy On Tue, Feb 25, 2020 at 6:18 AM Steve Brasier <[email protected]> wrote: > Hi all, > > I'm trying to configure a lustre 2.12.2 system with SSK and I believe I > have a problem with kernel keys which I'd be grateful for any suggestions > on. > > Essentially I have followed the instructions/examples in the docs for SSK > except that: > - A host "lustre-storage" hosts the MGS, MDT and OST with the fileystem > "test_fs1". > - A host "lustre-client1" is in a nodeset "lustre_client1". > - cli2ost and cli2mdt rules set as skn > (happy to provide more details if required but I think that's the major > differences from the examples) > > Trying to mount the filesystem from the client fails: > > [centos@lustre-client1 ~]$ sudo mount -t lustre 192.168.41.10@tcp1:/test_fs1 > -o skpath=/etc/lustre /mnt/lustre/test_fs1/ > mount.lustre: mount 192.168.41.10@tcp1:/test_fs1 at /mnt/lustre/test_fs1 > failed: Connection refused > > Looking in /var/log/messages I can see: > > lustre-client1 lgss_keyring: [18756]:ERROR:sk_create_cred(): keyctl_read() > failed for key 1073636326: Permission denied > > And in fact there is a problem reading this key: > [centos@lustre-client1 ~]$ sudo keyctl list @u > 1 key in keyring: > 1073636326: --alswrv 0 0 user: lustre:test_fs1 > [centos@lustre-client1 ~]$ sudo keyctl read 1073636326 > keyctl_read_alloc: Permission denied > > If I try to create a key myself I can see it has the same permissions as > 1073636326, and again reading it fails. Some googling led me to this > <https://mjg59.dreamwidth.org/37333.html> which suggests there's a > fundamental problem using sudo with kernel keys *. I can't be the only > person to try to deploy lustre using sudo though surely? So there must be > something I'm missing here to make this work. > > To work around this I tried including "user" in the /etc/fstab options > then mounting as a normal user but that fails: > [centos@lustre-client1 ~]$ mount /mnt/lustre/test_fs1/ > mount.lustre: mount 192.168.41.10@tcp1:/test_fs1 at /mnt/lustre/test_fs1 > failed: Operation not permitted > > and in fact it appears lustre doesn't support the user option? > Feb 25 11:12:27 lustre-client1 kernel: LustreError: 152-6: Unknown option > 'user', won't mount. > > > As I said any help appreciated! > > Steve > > * Although that link says key possession is tied to the original user, > which would suggest that the key should show up in centos's keyring, which > it doesn't. > > http://stackhpc.com/ > Please note I work Tuesday to Friday. > >> _______________________________________________ > lustre-discuss mailing list > [email protected] > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org >
_______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
