Re: sem_lock() vs qspinlocks

2016-05-24 Thread Boqun Feng
On Mon, May 23, 2016 at 10:52:09AM -0700, Linus Torvalds wrote: > On Mon, May 23, 2016 at 5:25 AM, Peter Zijlstra wrote: > > > > Paul has smp_mb__after_unlock_lock() for the RCpc 'upgrade'. How about > > something like: > > > > smp_mb__after_lock() > > I'd much rather make the naming be h

Re: sem_lock() vs qspinlocks

2016-05-24 Thread Peter Zijlstra
On Sat, May 21, 2016 at 03:49:20PM +0200, Manfred Spraul wrote: > >I'm tempted to put that trailing smp_rmb() in spin_unlock_wait() too; > >because I suspect the netfilter code is broken without it. > > > >And it seems intuitive to assume that if we return from unlock_wait() we > >can indeed obser

Re: sem_lock() vs qspinlocks

2016-05-23 Thread Linus Torvalds
On Mon, May 23, 2016 at 5:25 AM, Peter Zijlstra wrote: > > Paul has smp_mb__after_unlock_lock() for the RCpc 'upgrade'. How about > something like: > > smp_mb__after_lock() I'd much rather make the naming be higher level. It's not necessarily going to be a "mb", and while the problem is a

Re: sem_lock() vs qspinlocks

2016-05-23 Thread Peter Zijlstra
On Fri, May 20, 2016 at 10:00:45AM -0700, Linus Torvalds wrote: > So I do wonder if we should make that smp_mb() be something the > *caller* has to do, and document rules for it. IOW, introduce a new > spinlock primitive called "spin_lock_synchronize()", and then spinlock > implementations that ha

Re: sem_lock() vs qspinlocks

2016-05-22 Thread Peter Zijlstra
On Sun, May 22, 2016 at 10:43:08AM +0200, Manfred Spraul wrote: > How would we handle mixed spin_lock()/mutex_lock() code? > For the IPC code, I would like to replace the outer lock with a mutex. > The code only uses spinlocks, because at the time it was written, the mutex > code didn't contain a b

Re: sem_lock() vs qspinlocks

2016-05-22 Thread Manfred Spraul
Hi Peter, On 05/20/2016 06:04 PM, Peter Zijlstra wrote: On Fri, May 20, 2016 at 05:21:49PM +0200, Peter Zijlstra wrote: Let me write a patch.. OK, something like the below then.. lemme go build that and verify that too fixes things. --- Subject: locking,qspinlock: Fix spin_is_locked() and s

Re: sem_lock() vs qspinlocks

2016-05-21 Thread Davidlohr Bueso
On Sat, 21 May 2016, Peter Zijlstra wrote: On Fri, May 20, 2016 at 05:48:39PM -0700, Davidlohr Bueso wrote: On Fri, 20 May 2016, Linus Torvalds wrote: >Oh, I definitely agree on the stable part, and yes, the "splt things >up" model should come later if people agree that it's a good thing. Th

Re: sem_lock() vs qspinlocks

2016-05-21 Thread Manfred Spraul
On 05/21/2016 09:37 AM, Peter Zijlstra wrote: On Fri, May 20, 2016 at 05:48:39PM -0700, Davidlohr Bueso wrote: As opposed to spin_is_locked(), spin_unlock_wait() is perhaps more tempting to use for locking correctness. For example, taking a look at nf_conntrack_all_lock(), it too likes to get s

Re: sem_lock() vs qspinlocks

2016-05-21 Thread Peter Zijlstra
On Sat, May 21, 2016 at 12:01:00AM -0400, Waiman Long wrote: > On 05/20/2016 08:59 PM, Davidlohr Bueso wrote: > >On Fri, 20 May 2016, Peter Zijlstra wrote: > > > >>On Fri, May 20, 2016 at 04:47:43PM -0400, Waiman Long wrote: > >> > Similarly, and I know you hate it, but afaict, then semanticall

Re: sem_lock() vs qspinlocks

2016-05-21 Thread Peter Zijlstra
On Fri, May 20, 2016 at 05:48:39PM -0700, Davidlohr Bueso wrote: > On Fri, 20 May 2016, Linus Torvalds wrote: > > > >Oh, I definitely agree on the stable part, and yes, the "splt things > >up" model should come later if people agree that it's a good thing. > > The backporting part is quite nice,

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Waiman Long
On 05/20/2016 08:59 PM, Davidlohr Bueso wrote: On Fri, 20 May 2016, Peter Zijlstra wrote: On Fri, May 20, 2016 at 04:47:43PM -0400, Waiman Long wrote: >Similarly, and I know you hate it, but afaict, then semantically >queued_spin_is_contended() ought to be: > >- return atomic_read(&lock

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Linus Torvalds
On Fri, May 20, 2016 at 5:48 PM, Davidlohr Bueso wrote: > > I can verify that this patch fixes the issue. Ok, I've applied it to my tree. Linus

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Davidlohr Bueso
On Fri, 20 May 2016, Peter Zijlstra wrote: On Fri, May 20, 2016 at 04:47:43PM -0400, Waiman Long wrote: >Similarly, and I know you hate it, but afaict, then semantically >queued_spin_is_contended() ought to be: > >- return atomic_read(&lock->val) & ~_Q_LOCKED_MASK; >+ return atomic

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Davidlohr Bueso
On Fri, 20 May 2016, Linus Torvalds wrote: Oh, I definitely agree on the stable part, and yes, the "splt things up" model should come later if people agree that it's a good thing. The backporting part is quite nice, yes, but ultimately I think I prefer Linus' suggestion making things explicit

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Linus Torvalds
On Fri, May 20, 2016 at 2:06 PM, Peter Zijlstra wrote: >> >> See for example "ipc_smp_acquire__after_spin_is_unlocked()", which has >> a big comment atop of it that now becomes nonsensical with this patch. > > Not quite; we still need that I think. I think so too, but it's the *comment* that is n

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 10:00:45AM -0700, Linus Torvalds wrote: > On Fri, May 20, 2016 at 9:04 AM, Peter Zijlstra wrote: > > +* queued_spin_lock_slowpath() can ACQUIRE the lock before > > +* issuing the unordered store that sets _Q_LOCKED_VAL. > > Ugh. This was my least favorite p

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 04:44:19PM -0400, Waiman Long wrote: > On 05/20/2016 07:58 AM, Peter Zijlstra wrote: > >On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > >>As such, the following restores the behavior of the ticket locks and 'fixes' > >>(or hides?) the bug in sems. Naturall

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 04:47:43PM -0400, Waiman Long wrote: > >Similarly, and I know you hate it, but afaict, then semantically > >queued_spin_is_contended() ought to be: > > > >- return atomic_read(&lock->val) & ~_Q_LOCKED_MASK; > >+ return atomic_read(&lock->val); > > > Looking for

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Waiman Long
On 05/20/2016 11:00 AM, Davidlohr Bueso wrote: On Fri, 20 May 2016, Peter Zijlstra wrote: On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: In addition, this makes me wonder if queued_spin_is_locked() should then be: -return atomic_read(&lock->val); +return atomic_rea

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Waiman Long
On 05/20/2016 07:58 AM, Peter Zijlstra wrote: On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: As such, the following restores the behavior of the ticket locks and 'fixes' (or hides?) the bug in sems. Naturally incorrect approach: @@ -290,7 +290,8 @@ static void sem_wait_array(s

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Linus Torvalds
On Fri, May 20, 2016 at 9:04 AM, Peter Zijlstra wrote: > +* queued_spin_lock_slowpath() can ACQUIRE the lock before > +* issuing the unordered store that sets _Q_LOCKED_VAL. Ugh. This was my least favorite part of the queued locks, and I really liked the completely unambiguous sem

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Davidlohr Bueso
On Fri, 20 May 2016, Peter Zijlstra wrote: The problem is that the clear_pending_set_locked() is an unordered store, therefore this store can be delayed until no later than spin_unlock() (which orders against it due to the address dependency). This opens numerous races; for example: ip

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 05:21:49PM +0200, Peter Zijlstra wrote: > Let me write a patch.. OK, something like the below then.. lemme go build that and verify that too fixes things. --- Subject: locking,qspinlock: Fix spin_is_locked() and spin_unlock_wait() Similar to commits: 51d7d5205d33 ("po

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 05:05:05PM +0200, Peter Zijlstra wrote: > On Fri, May 20, 2016 at 08:00:49AM -0700, Davidlohr Bueso wrote: > > On Fri, 20 May 2016, Peter Zijlstra wrote: > > > > >On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > > >> In addition, this makes me wonder if qu

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Davidlohr Bueso
On Fri, 20 May 2016, Peter Zijlstra wrote: On Fri, May 20, 2016 at 08:00:49AM -0700, Davidlohr Bueso wrote: On Fri, 20 May 2016, Peter Zijlstra wrote: >On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: >> In addition, this makes me wonder if queued_spin_is_locked() should then b

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 10:05:33PM +0800, Boqun Feng wrote: > On Fri, May 20, 2016 at 01:58:19PM +0200, Peter Zijlstra wrote: > > On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > > > As such, the following restores the behavior of the ticket locks and > > > 'fixes' > > > (or hide

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 08:00:49AM -0700, Davidlohr Bueso wrote: > On Fri, 20 May 2016, Peter Zijlstra wrote: > > >On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > >> In addition, this makes me wonder if queued_spin_is_locked() should then > >> be: > >> > >>- return atomic_rea

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Davidlohr Bueso
On Fri, 20 May 2016, Peter Zijlstra wrote: On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: In addition, this makes me wonder if queued_spin_is_locked() should then be: - return atomic_read(&lock->val); + return atomic_read(&lock->val) & _Q_LOCKED_MASK; And avoid

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Boqun Feng
Hi Peter, On Fri, May 20, 2016 at 01:58:19PM +0200, Peter Zijlstra wrote: > On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > > As such, the following restores the behavior of the ticket locks and 'fixes' > > (or hides?) the bug in sems. Naturally incorrect approach: > > > > @@ -

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > As such, the following restores the behavior of the ticket locks and 'fixes' > (or hides?) the bug in sems. Naturally incorrect approach: > > @@ -290,7 +290,8 @@ static void sem_wait_array(struct sem_array *sma) > > for (i =

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Mel Gorman
On Fri, May 20, 2016 at 12:09:01PM +0200, Ingo Molnar wrote: > > * Peter Zijlstra wrote: > > > On Fri, May 20, 2016 at 10:30:08AM +0200, Peter Zijlstra wrote: > > > On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > > > > Specifically > > > > for the 'cascade_cond' and 'cascade_f

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Ingo Molnar
* Peter Zijlstra wrote: > On Fri, May 20, 2016 at 10:30:08AM +0200, Peter Zijlstra wrote: > > On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > > > Specifically > > > for the 'cascade_cond' and 'cascade_flock' programs, which exhibit hangs > > > in libc's > > > semop() blocked

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 11:07:49AM +0200, Giovanni Gherdovich wrote: > --- run_cascade.sh - > #!/bin/bash > > TESTCASE=$1 > CASCADE_PATH="libmicro-1-installed/bin-x86_64" > > case $TESTCASE in > c_flock_200) > BINNAME="cascade_flock" >

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 10:30:08AM +0200, Peter Zijlstra wrote: > On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > > Specifically > > for the 'cascade_cond' and 'cascade_flock' programs, which exhibit hangs in > > libc's > > semop() blocked waiting for zero. > > OK; so I've been

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > Specifically > for the 'cascade_cond' and 'cascade_flock' programs, which exhibit hangs in > libc's > semop() blocked waiting for zero. OK; so I've been running: while :; do bin/cascade_cond -E -C 200 -L -S -W -T 200 -I 20

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Fri, May 20, 2016 at 10:13:15AM +0200, Peter Zijlstra wrote: > On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > > > [1] https://hg.java.net/hg/libmicro~hg-repo > > So far I've managed to install mercurial and clone this thing, but it > doesn't actually build :/ > > I'll try

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > [1] https://hg.java.net/hg/libmicro~hg-repo So far I've managed to install mercurial and clone this thing, but it doesn't actually build :/ I'll try harder..

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > However, this is semantically different to > what was previously done with ticket locks in that spin_unlock_wait() will > always observe > all waiters by adding itself to the tail. static inline void arch_spin_unlock_wait(arch_s

Re: sem_lock() vs qspinlocks

2016-05-20 Thread Peter Zijlstra
On Thu, May 19, 2016 at 10:39:26PM -0700, Davidlohr Bueso wrote: > In addition, this makes me wonder if queued_spin_is_locked() should then be: > > - return atomic_read(&lock->val); > + return atomic_read(&lock->val) & _Q_LOCKED_MASK; > > And avoid considering pending waiters as locked.