Mateusz, why do you __predict_false() the recursion scenario? I'm afraid that performance loss for mispredictions could outweight the gain due to predictions. AFAIK, mutex recursion is still a pretty common event in the kernel.
On Mon, Feb 20, 2017 at 07:08:36PM +0000, Mateusz Guzik wrote: M> Author: mjg M> Date: Mon Feb 20 19:08:36 2017 M> New Revision: 313996 M> URL: https://svnweb.freebsd.org/changeset/base/313996 M> M> Log: M> mtx: fix spin mutexes interaction with failed fcmpset M> M> While doing so move recursion support down to the fallback routine. M> M> Modified: M> head/sys/kern/kern_mutex.c M> head/sys/sys/mutex.h M> M> Modified: head/sys/kern/kern_mutex.c M> ============================================================================== M> --- head/sys/kern/kern_mutex.c Mon Feb 20 17:33:25 2017 (r313995) M> +++ head/sys/kern/kern_mutex.c Mon Feb 20 19:08:36 2017 (r313996) M> @@ -696,6 +696,14 @@ _mtx_lock_spin_cookie(volatile uintptr_t M> lock_delay_arg_init(&lda, &mtx_spin_delay); M> m = mtxlock2mtx(c); M> M> + if (__predict_false(v == MTX_UNOWNED)) M> + v = MTX_READ_VALUE(m); M> + M> + if (__predict_false(v == tid)) { M> + m->mtx_recurse++; M> + return; M> + } M> + M> if (LOCK_LOG_TEST(&m->lock_object, opts)) M> CTR1(KTR_LOCK, "_mtx_lock_spin: %p spinning", m); M> KTR_STATE1(KTR_SCHED, "thread", sched_tdname((struct thread *)tid), M> M> Modified: head/sys/sys/mutex.h M> ============================================================================== M> --- head/sys/sys/mutex.h Mon Feb 20 17:33:25 2017 (r313995) M> +++ head/sys/sys/mutex.h Mon Feb 20 19:08:36 2017 (r313996) M> @@ -223,12 +223,9 @@ void thread_lock_flags_(struct thread *, M> uintptr_t _v = MTX_UNOWNED; \ M> \ M> spinlock_enter(); \ M> - if (!_mtx_obtain_lock_fetch((mp), &_v, _tid)) { \ M> - if (_v == _tid) \ M> - (mp)->mtx_recurse++; \ M> - else \ M> - _mtx_lock_spin((mp), _v, _tid, (opts), (file), (line));\ M> - } else \ M> + if (!_mtx_obtain_lock_fetch((mp), &_v, _tid)) \ M> + _mtx_lock_spin((mp), _v, _tid, (opts), (file), (line)); \ M> + else \ M> LOCKSTAT_PROFILE_OBTAIN_LOCK_SUCCESS(spin__acquire, \ M> mp, 0, 0, file, line); \ M> } while (0) M> _______________________________________________ M> svn-src-...@freebsd.org mailing list M> https://lists.freebsd.org/mailman/listinfo/svn-src-all M> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org" -- Totus tuus, Glebius. _______________________________________________ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
