On Sat, Mar 02, 2024 at 05:40:08AM -0800, Rick Macklem wrote: > On Fri, Mar 1, 2024 at 10:51 PM Konstantin Belousov <k...@freebsd.org> wrote: > > > > CAUTION: This email originated from outside of the University of Guelph. Do > > not click links or open attachments unless you recognize the sender and > > know the content is safe. If in doubt, forward suspicious emails to > > ith...@uoguelph.ca. > > > > > > On Fri, Mar 01, 2024 at 06:23:56AM -0800, Rick Macklem wrote: > > > On Fri, Mar 1, 2024 at 12:00 AM Ronald Klop <ronald-li...@klop.ws> wrote: > > > > > > > > Interesting read. > > > > > > > > Would it be possible to separate locking for admin actions like a > > > > client mounting an fs from traffic flowing for file operations? > > > Well, the NFS server does not really have any concept of a mount. > > > What I am referring to is the ClientID maintained for NFSv4 mounts, > > > which all the open/lock/session/layout state hangs off of. > > > > > > For most cases, this state information can safely be accessed/modified > > > via a mutex, but there are three exceptions: > > > - creating a new ClientID (which is done by the ExchangeID operation) > > > and typically happens when a NFS client does a mount. > > > - delegation Recall (which only happens when delegations are enabled) > > > One of the reasons delegations are not enabled by default on the > > > FreeBSD server. > > > - the DestroyClientID which is typically done by a NFS client during > > > dismount. > > > For these cases, it is just too difficult to do them without sleeping. > > > As such, there is a sleep lock which the nfsd threads normally acquire > > > shared > > > when doing NFSv4 operations, but for the above cases the lock is aquired > > > exclusive. > > > - I had to give the exclusive lock priority over shared lock > > > acquisition (it is a > > > custom locking mechanism with assorted weirdnesses) because without > > > that someone reported that new mounts took up to 1/2hr to occur. > > > (The exclusive locker waited for 30min before all the other nfsd threads > > > were not busy.) > > > Because of this priority, once a nfsd thread requests the exclusive > > > lock, > > > all other nfsd threads executing NFSv4 RPCs block after releasing their > > > shared lock, until the exclusive locker releases the exclusive lock. > > Normal lockmgr locks + TDP_DEADLKTREAT private thread flag provide the > > property of pref. exclusive waiters in presence of the shared waiters. > > I think this is what you described above. > It also has some weird properties, like if there are multiple requestors > for the exclusive lock, once one thread gets it (the threads are nfsd worker > threads and indistinct), the others that requested an exclusive thread are > unblocked without the lock being issued to them. This sounds to me as LK_SLEEPFAIL feature of lockmgr. Do not underestimate the amount of weird features in it.
> They then check if the exclusive lock is still needed (usually not, since > the other thread has dealt with the case where it was needed) and > then they can acquire a shared lock. > Without this, there were cases where several threads would acquire > the exclusive lock and then discover that the lock was not needed and > just release it again. > > It also uses an assortment of weird flags/call args. > > rick >
