On Tue, Jul 14, 2015 at 10:13:32PM -0400, Waiman Long wrote: > The smp_store_release() is not a full barrier. In order to avoid missed > wakeup, we may need to add memory barrier around locked and cpu state > variables adding to complexity. As the chance of spurious wakeup is very > low, it is easier and safer to just do an unconditional kick at unlock > time.
I have the below patch. We need that rmb in there anyhow for the hash to work. --- Subject: locking/pvqspinlock: Order pv_unhash after cmpxchg on unlock slowpath From: Will Deacon <will.dea...@arm.com> Date: Mon, 13 Jul 2015 16:58:30 +0100 When we unlock in __pv_queued_spin_unlock, a failed cmpxchg on the lock value indicates that we need to take the slow-path and unhash the corresponding node blocked on the lock. Since a failed cmpxchg does not provide any memory-ordering guarantees, it is possible that the node data could be read before the cmpxchg on weakly-ordered architectures and therefore return a stale value, leading to hash corruption and/or a BUG(). This patch adds an smb_rmb() following the failed cmpxchg operation, so that the unhashing is ordered after the lock has been checked. Cc: Paul McKenney <paul...@linux.vnet.ibm.com> Cc: Waiman Long <waiman.l...@hp.com> Cc: Steve Capper <steve.cap...@arm.com> Reported-by: Peter Zijlstra <pet...@infradead.org> Signed-off-by: Will Deacon <will.dea...@arm.com> [peterz: More comments] Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org> Link: http://lkml.kernel.org/r/20150713155830.gl2...@arm.com --- kernel/locking/qspinlock_paravirt.h | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) --- a/kernel/locking/qspinlock_paravirt.h +++ b/kernel/locking/qspinlock_paravirt.h @@ -244,13 +244,17 @@ static void pv_wait_head(struct qspinloc if (!lp) { /* ONCE */ lp = pv_hash(lock, pn); /* - * lp must be set before setting _Q_SLOW_VAL + * We must hash before setting _Q_SLOW_VAL, such that + * when we observe _Q_SLOW_VAL in __pv_queued_spin_unlock() + * we'll be sure to be able to observe our hash entry. * - * [S] lp = lock [RmW] l = l->locked = 0 - * MB MB - * [S] l->locked = _Q_SLOW_VAL [L] lp + * [S] pn->state + * [S] <hash> [Rmw] l->locked == _Q_SLOW_VAL + * MB RMB + * [RmW] l->locked = _Q_SLOW_VAL [L] <unhash> + * [L] pn->state * - * Matches the cmpxchg() in __pv_queued_spin_unlock(). + * Matches the smp_rmb() in __pv_queued_spin_unlock(). */ if (!cmpxchg(&l->locked, _Q_LOCKED_VAL, _Q_SLOW_VAL)) { /* @@ -306,6 +310,15 @@ __visible void __pv_queued_spin_unlock(s } /* + * A failed cmpxchg doesn't provide any memory-ordering guarantees, + * so we need a barrier to order the read of the node data in + * pv_unhash *after* we've read the lock being _Q_SLOW_VAL. + * + * Matches the cmpxchg() in pv_wait_head() setting _Q_SLOW_VAL. + */ + smp_rmb(); + + /* * Since the above failed to release, this must be the SLOW path. * Therefore start by looking up the blocked node and unhashing it. */ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
