Re: Subject: Warning in workqueue.c

2014-03-17 Thread Peter Zijlstra
On Mon, Mar 10, 2014 at 10:37:42AM -0400, Jason J. Herne wrote: > > Did you intend for me to run with this patch or was it posted for discussion > only? If you want it run, please tell me what to look for. > Also, if I should run this, should I include any other patches, either the > last one you

Re: Subject: Warning in workqueue.c

2014-03-17 Thread Jason J. Herne
On 03/10/2014 10:37 AM, Jason J. Herne wrote: On 02/25/2014 05:37 AM, Peter Zijlstra wrote: On Mon, Feb 24, 2014 at 01:35:01PM -0500, Tejun Heo wrote: That's a bummer but it at least isn't a very new regression. Peter, any ideas on debugging this? I can make workqueue to play block / unblock

Re: Subject: Warning in workqueue.c

2014-03-10 Thread Jason J. Herne
On 02/25/2014 05:37 AM, Peter Zijlstra wrote: On Mon, Feb 24, 2014 at 01:35:01PM -0500, Tejun Heo wrote: That's a bummer but it at least isn't a very new regression. Peter, any ideas on debugging this? I can make workqueue to play block / unblock dance to try to work around the issue but that

Re: Subject: Warning in workqueue.c

2014-02-25 Thread Peter Zijlstra
On Mon, Feb 24, 2014 at 01:35:01PM -0500, Tejun Heo wrote: > That's a bummer but it at least isn't a very new regression. Peter, > any ideas on debugging this? I can make workqueue to play block / > unblock dance to try to work around the issue but that'd be very > yucky. It'd be great to root

Re: Subject: Warning in workqueue.c

2014-02-24 Thread Tejun Heo
On Mon, Feb 24, 2014 at 10:01:55AM -0500, Jason J. Herne wrote: > On 02/14/2014 11:25 AM, Tejun Heo wrote: > >... > >Hmmm... weird, p's rq shouldn't have changed without its cpus_allowed > >busted. Anyways, let's wait for Jason's test results and see whether > >this is a regression at all. > > I

Re: Subject: Warning in workqueue.c

2014-02-24 Thread Jason J. Herne
On 02/14/2014 11:25 AM, Tejun Heo wrote: ... Hmmm... weird, p's rq shouldn't have changed without its cpus_allowed busted. Anyways, let's wait for Jason's test results and see whether this is a regression at all. I was unable to determine exactly when this behavior was introduced. The reason

Re: Subject: Warning in workqueue.c

2014-02-14 Thread Tejun Heo
On Fri, Feb 14, 2014 at 05:28:54PM +0100, Peter Zijlstra wrote: > On Fri, Feb 14, 2014 at 11:25:56AM -0500, Tejun Heo wrote: > > Hey, we now even keep normal kthreads across cpu down/ups. :) > > Well, we keep them, but parked, they're not allowed to run. Yeah, that's true. In the long term, I th

Re: Subject: Warning in workqueue.c

2014-02-14 Thread Peter Zijlstra
On Fri, Feb 14, 2014 at 11:25:56AM -0500, Tejun Heo wrote: > Hey, we now even keep normal kthreads across cpu down/ups. :) Well, we keep them, but parked, they're not allowed to run. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vg

Re: Subject: Warning in workqueue.c

2014-02-14 Thread Tejun Heo
Hey, Peter. On Fri, Feb 14, 2014 at 05:09:23PM +0100, Peter Zijlstra wrote: > > Ingo, Peter, Jason is reporting workqueue triggering warning because a > > worker is running on the wrong CPU, which is relatively reliably > > reproducible with the above workload on s390. > > Wasn't that a feature

Re: Subject: Warning in workqueue.c

2014-02-14 Thread Peter Zijlstra
On Thu, Feb 13, 2014 at 03:41:02PM -0500, Tejun Heo wrote: > Hello, > > (cc'ing Ingo and Peter) > > On Thu, Feb 13, 2014 at 12:58:10PM -0500, Jason J. Herne wrote: > > [ 5779.795687] [ cut here ] > > [ 5779.795695] WARNING: at kernel/workqueue.c:2159 > > > [ 5779.7958

Re: Subject: Warning in workqueue.c

2014-02-14 Thread Tejun Heo
Hello, Jason. On Fri, Feb 14, 2014 at 9:56 AM, Jason J. Herne wrote: > I will try to find a kernel where this is not reproducible and attempt > bisect from there. I will let you know what I find. Yes, even just finding that there's an earlier version where this doesn't happen would be great. Th

Re: Subject: Warning in workqueue.c

2014-02-14 Thread Jason J. Herne
On 02/13/2014 03:41 PM, Tejun Heo wrote: Jason, I don't have much idea from workqueue side. Have you been running this test with older kernels too? Can you confirm whether this failure is something recent? Bisection would be awesome but just confirming, say, 3.12 doesn't have this issue would

Re: Subject: Warning in workqueue.c

2014-02-13 Thread Tejun Heo
Hello, (cc'ing Ingo and Peter) On Thu, Feb 13, 2014 at 12:58:10PM -0500, Jason J. Herne wrote: > [ 5779.795687] [ cut here ] > [ 5779.795695] WARNING: at kernel/workqueue.c:2159 > [ 5779.795844] XXX: worker->flags=0x1 pool->flags=0x0 cpu=4 pool->cpu=5(1) > rescue_wq=

Re: Subject: Warning in workqueue.c

2014-02-13 Thread Jason J. Herne
On 02/12/2014 10:31 PM, Lai Jiangshan wrote: On 02/12/2014 11:18 PM, Jason J. Herne wrote: Could you use the following patch for test if Tejun doesn't give you a new one. Lai, Here is the output using the patch you asked me to run with. [ 5779.795687] [ cut here ] [

Re: Subject: Warning in workqueue.c

2014-02-12 Thread Lai Jiangshan
On 02/12/2014 11:18 PM, Jason J. Herne wrote: > On 02/10/2014 06:17 PM, Tejun Heo wrote: >> Hello, >> >> On Mon, Feb 10, 2014 at 10:32:11AM -0500, Jason J. Herne wrote: >>> [ 950.778485] XXX: worker->flags=0x1 pool->flags=0x0 cpu=6 >>> pool->cpu=2 rescue_wq= (null) >>> [ 950.778488] XXX:

Re: Subject: Warning in workqueue.c

2014-02-12 Thread Lai Jiangshan
On 02/12/2014 11:18 PM, Jason J. Herne wrote: > On 02/10/2014 06:17 PM, Tejun Heo wrote: >> Hello, >> >> On Mon, Feb 10, 2014 at 10:32:11AM -0500, Jason J. Herne wrote: >>> [ 950.778485] XXX: worker->flags=0x1 pool->flags=0x0 cpu=6 >>> pool->cpu=2 rescue_wq= (null) >>> [ 950.778488] XXX:

Re: Subject: Warning in workqueue.c

2014-02-12 Thread Jason J. Herne
On 02/10/2014 06:17 PM, Tejun Heo wrote: Hello, On Mon, Feb 10, 2014 at 10:32:11AM -0500, Jason J. Herne wrote: [ 950.778485] XXX: worker->flags=0x1 pool->flags=0x0 cpu=6 pool->cpu=2 rescue_wq= (null) [ 950.778488] XXX: last_unbind=-7 last_rebind=0 last_rebound_clear=0 nr_exected_aft

Re: Subject: Warning in workqueue.c

2014-02-10 Thread Tejun Heo
Hello, On Mon, Feb 10, 2014 at 10:32:11AM -0500, Jason J. Herne wrote: > [ 950.778485] XXX: worker->flags=0x1 pool->flags=0x0 cpu=6 > pool->cpu=2 rescue_wq= (null) > [ 950.778488] XXX: last_unbind=-7 last_rebind=0 > last_rebound_clear=0 nr_exected_after_rebound_clear=0 > [ 950.778492]

Re: Subject: Warning in workqueue.c

2014-02-10 Thread Jason J. Herne
On 02/07/2014 02:36 PM, Tejun Heo wrote: Hello, On Fri, Feb 07, 2014 at 12:55:28PM -0500, Jason J. Herne wrote: [ 644.517710] XXX: worker->flags=0x1 pool->flags=0x0 cpu=6 pool->cpu=4 [ 731.367023] XXX: worker->flags=0x1 pool->flags=0x0 cpu=1 pool->cpu=7 Sorry, still no idea how this can hap

Re: Subject: Warning in workqueue.c

2014-02-07 Thread Tejun Heo
Hello, On Fri, Feb 07, 2014 at 12:55:28PM -0500, Jason J. Herne wrote: > [ 644.517710] XXX: worker->flags=0x1 pool->flags=0x0 cpu=6 pool->cpu=4 > [ 731.367023] XXX: worker->flags=0x1 pool->flags=0x0 cpu=1 pool->cpu=7 Sorry, still no idea how this can happen. Can you please try the following?

Re: Subject: Warning in workqueue.c

2014-02-07 Thread Jason J. Herne
On 02/07/2014 11:51 AM, Tejun Heo wrote: Hello, (cc'ing Lai as he knows a lot of workqueue code and quoting the whole body for him) Hmmm my memory is a bit rusty and nothing rings a bell immediately. Can you please try the patch at the end of this message and report the debug message? Let

Re: Subject: Warning in workqueue.c

2014-02-07 Thread Tejun Heo
Hello, (cc'ing Lai as he knows a lot of workqueue code and quoting the whole body for him) Hmmm my memory is a bit rusty and nothing rings a bell immediately. Can you please try the patch at the end of this message and report the debug message? Let's first find out what's going on. Thanks