Re: vmstat: On demand vmstat workers V8

2014-08-06 Thread Sasha Levin
On 08/06/2014 10:12 AM, Christoph Lameter wrote: > On Tue, 5 Aug 2014, Sasha Levin wrote: > >> > I can easily trigger it by cranking up the cpu hotplug code. Just try to >> > frequently offline and online cpus, it should reproduce quickly. > Thats what I thought. > > The test was done with this f

Re: vmstat: On demand vmstat workers V8

2014-08-06 Thread Christoph Lameter
On Tue, 5 Aug 2014, Sasha Levin wrote: > I can easily trigger it by cranking up the cpu hotplug code. Just try to > frequently offline and online cpus, it should reproduce quickly. Thats what I thought. The test was done with this fix applied right? Subject: vmstat ondemand: Fix online/offline

Re: vmstat: On demand vmstat workers V8

2014-08-05 Thread Sasha Levin
On 08/05/2014 10:51 AM, Christoph Lameter wrote: > On Mon, 4 Aug 2014, Sasha Levin wrote: > >> On 07/25/2014 10:22 PM, Sasha Levin wrote: >>> On 07/10/2014 10:04 AM, Christoph Lameter wrote: > This patch creates a vmstat shepherd worker that monitors the > per cpu differentials on all proc

Re: vmstat: On demand vmstat workers V8

2014-08-05 Thread Christoph Lameter
On Mon, 4 Aug 2014, Sasha Levin wrote: > On 07/25/2014 10:22 PM, Sasha Levin wrote: > > On 07/10/2014 10:04 AM, Christoph Lameter wrote: > >> > This patch creates a vmstat shepherd worker that monitors the > >> > per cpu differentials on all processors. If there are differentials > >> > on a proce

Re: vmstat: On demand vmstat workers V8

2014-08-04 Thread Sasha Levin
On 07/25/2014 10:22 PM, Sasha Levin wrote: > On 07/10/2014 10:04 AM, Christoph Lameter wrote: >> > This patch creates a vmstat shepherd worker that monitors the >> > per cpu differentials on all processors. If there are differentials >> > on a processor then a vmstat worker local to the processors

Re: vmstat: On demand vmstat workers V8

2014-07-30 Thread Lai Jiangshan
On 07/30/2014 10:45 PM, Christoph Lameter wrote: > On Wed, 30 Jul 2014, Lai Jiangshan wrote: > >> I think the bug is here, it re-queues the per_cpu(vmstat_work, cpu) which is >> offline >> (after vmstat_cpuup_callback(CPU_DOWN_PREPARE). And cpu_stat_off is >> accessed without >> proper lock. >

Re: vmstat: On demand vmstat workers V8

2014-07-30 Thread Christoph Lameter
On Wed, 30 Jul 2014, Lai Jiangshan wrote: > I think the bug is here, it re-queues the per_cpu(vmstat_work, cpu) which is > offline > (after vmstat_cpuup_callback(CPU_DOWN_PREPARE). And cpu_stat_off is accessed > without > proper lock. Ok. I guess we need to make the preemption check output mor

Re: vmstat: On demand vmstat workers V8

2014-07-30 Thread Christoph Lameter
On Wed, 30 Jul 2014, Lai Jiangshan wrote: > > > > > > Index: linux/mm/vmstat.c > > === > > --- linux.orig/mm/vmstat.c 2014-07-29 10:22:45.073884943 -0500 > > +++ linux/mm/vmstat.c 2014-07-29 10:34:45.083369228 -0500 > > @@ -127

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Lai Jiangshan
On 07/29/2014 11:39 PM, Christoph Lameter wrote: > On Tue, 29 Jul 2014, Tejun Heo wrote: > >> Hmmm, well, then it's something else. Either a bug in workqueue or in >> the caller. Given the track record, the latter is more likely. >> e.g. it looks kinda suspicious that the work func is cleared af

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Lai Jiangshan
On 07/11/2014 11:17 PM, Christoph Lameter wrote: > On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > >>> Converted what? We still need to keep a cpumask around that tells us which >>> processor have vmstat running and which do not. >>> >> >> Converted to cpumask_var_t. >> >> I mean we spent dozens

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Lai Jiangshan
If I understand the semantics of the cpu_stat_off correctly, please read. cpu_stat_off = a set of such CPU: the cpu is online && vmstat_work is off I consider some code forget to guarantee each cpu in cpu_stat_off is online. Thanks, Lai On 07/10/2014 10:04 PM, Christoph Lameter wrote: > + > +/*

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Christoph Lameter
On Tue, 29 Jul 2014, Sasha Levin wrote: > > Index: linux/mm/vmstat.c > > === > > --- linux.orig/mm/vmstat.c 2014-07-29 10:22:45.073884943 -0500 > > +++ linux/mm/vmstat.c 2014-07-29 10:34:45.083369228 -0500 > > @@ -1277,8 +1277,

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Sasha Levin
On 07/29/2014 11:39 AM, Christoph Lameter wrote: > On Tue, 29 Jul 2014, Tejun Heo wrote: > >> Hmmm, well, then it's something else. Either a bug in workqueue or in >> the caller. Given the track record, the latter is more likely. >> e.g. it looks kinda suspicious that the work func is cleared af

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Sasha Levin
On 07/29/2014 11:22 AM, Christoph Lameter wrote: > On Tue, 29 Jul 2014, Tejun Heo wrote: > >> I'm not sure that's a viable way forward. It's not like we can >> readily trigger the problematic cases which can lead to long pauses >> during cpu down. Besides, we need the distinction at the API leve

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Christoph Lameter
On Tue, 29 Jul 2014, Tejun Heo wrote: > Hmmm, well, then it's something else. Either a bug in workqueue or in > the caller. Given the track record, the latter is more likely. > e.g. it looks kinda suspicious that the work func is cleared after > cancel_delayed_work_sync() is called. What happen

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Christoph Lameter
On Tue, 29 Jul 2014, Tejun Heo wrote: > > mm/vmstat.c already has cpu down hooks. See vmstat_cpuup_callback(). > > Hmmm, well, then it's something else. Either a bug in workqueue or in > the caller. Given the track record, the latter is more likely. > e.g. it looks kinda suspicious that the work

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Christoph Lameter
On Tue, 29 Jul 2014, Tejun Heo wrote: > I'm not sure that's a viable way forward. It's not like we can > readily trigger the problematic cases which can lead to long pauses > during cpu down. Besides, we need the distinction at the API level, > which is the whole point of this. The best way pro

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Christoph Lameter
On Tue, 29 Jul 2014, Tejun Heo wrote: > I agree this can be improved, but at least for now, please add cpu > down hooks. We need them right now and they'll be helpful when later > separating out the correctness ones. mm/vmstat.c already has cpu down hooks. See vmstat_cpuup_callback(). -- To uns

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Christoph Lameter
On Mon, 28 Jul 2014, Andrew Morton wrote: > Also, Sasha's report showed this: > > [ 490.464613] kernel BUG at mm/vmstat.c:1278! > > That's your VM_BUG_ON() in vmstat_update(). That ain't no false > positive! AFAICT this is because of the earlier BUG(). -- To unsubscribe from this list: send t

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Tejun Heo
On Tue, Jul 29, 2014 at 10:10:11AM -0500, Christoph Lameter wrote: > On Tue, 29 Jul 2014, Tejun Heo wrote: > > > I agree this can be improved, but at least for now, please add cpu > > down hooks. We need them right now and they'll be helpful when later > > separating out the correctness ones. >

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Tejun Heo
(cc'ing Lai) Hello, On Tue, Jul 29, 2014 at 02:23:03PM +0200, Peter Zijlstra wrote: > > It's because we don't distinguish work items which are per-cpu for > > optimization and per-cpu for correctness and can't automatically flush > > / cancel / block per-cpu work items when a cpu goes down. I li

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Peter Zijlstra
On Tue, Jul 29, 2014 at 08:05:25AM -0400, Tejun Heo wrote: > On Tue, Jul 29, 2014 at 09:56:37AM +0200, Peter Zijlstra wrote: > > On Mon, Jul 28, 2014 at 01:55:17PM -0500, Christoph Lameter wrote: > > > On Fri, 25 Jul 2014, Sasha Levin wrote: > > > > > > > This patch doesn't interact well with my f

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Tejun Heo
On Tue, Jul 29, 2014 at 09:56:37AM +0200, Peter Zijlstra wrote: > On Mon, Jul 28, 2014 at 01:55:17PM -0500, Christoph Lameter wrote: > > On Fri, 25 Jul 2014, Sasha Levin wrote: > > > > > This patch doesn't interact well with my fuzzing setup. I'm seeing > > > the following: > > > > > > [ 490.4469

Re: vmstat: On demand vmstat workers V8

2014-07-29 Thread Peter Zijlstra
On Mon, Jul 28, 2014 at 01:55:17PM -0500, Christoph Lameter wrote: > On Fri, 25 Jul 2014, Sasha Levin wrote: > > > This patch doesn't interact well with my fuzzing setup. I'm seeing > > the following: > > > > [ 490.446927] BUG: using __this_cpu_read() in preemptible [] code: > > kworker/

Re: vmstat: On demand vmstat workers V8

2014-07-28 Thread Sasha Levin
On 07/28/2014 05:54 PM, Andrew Morton wrote: > Also, Sasha's report showed this: > > [ 490.464613] kernel BUG at mm/vmstat.c:1278! > > That's your VM_BUG_ON() in vmstat_update(). That ain't no false > positive! > > > > Is this code expecting that schedule_delayed_work() will schedule the > w

Re: vmstat: On demand vmstat workers V8

2014-07-28 Thread Andrew Morton
On Mon, 28 Jul 2014 13:55:17 -0500 (CDT) Christoph Lameter wrote: > On Fri, 25 Jul 2014, Sasha Levin wrote: > > > This patch doesn't interact well with my fuzzing setup. I'm seeing > > the following: > > > > [ 490.446927] BUG: using __this_cpu_read() in preemptible [] code: > > kworke

Re: vmstat: On demand vmstat workers V8

2014-07-28 Thread Christoph Lameter
On Fri, 25 Jul 2014, Sasha Levin wrote: > This patch doesn't interact well with my fuzzing setup. I'm seeing > the following: > > [ 490.446927] BUG: using __this_cpu_read() in preemptible [] code: > kworker/16:1/7368 > [ 490.447909] caller is __this_cpu_preempt_check+0x13/0x20 __this_c

Re: vmstat: On demand vmstat workers V8

2014-07-25 Thread Sasha Levin
On 07/10/2014 10:04 AM, Christoph Lameter wrote: > This patch creates a vmstat shepherd worker that monitors the > per cpu differentials on all processors. If there are differentials > on a processor then a vmstat worker local to the processors > with the differentials is created. That worker will

Re: vmstat: On demand vmstat workers V8

2014-07-14 Thread Christoph Lameter
On Mon, 14 Jul 2014, Hugh Dickins wrote: > On Fri, 11 Jul 2014, Christoph Lameter wrote: > > On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > > > > > Maybe just merge both? The whole looks good. > > > > I hope so. Andrew? > > I hope so, too: I know there are idle feckless^Htickless people > eager

Re: vmstat: On demand vmstat workers V8

2014-07-14 Thread Hugh Dickins
On Fri, 11 Jul 2014, Christoph Lameter wrote: > On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > > > Maybe just merge both? The whole looks good. > > I hope so. Andrew? I hope so, too: I know there are idle feckless^Htickless people eager for it. I did take the briefest of looks, but couldn't

Re: vmstat: On demand vmstat workers V8

2014-07-11 Thread Christoph Lameter
On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > Maybe just merge both? The whole looks good. I hope so. Andrew? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordo

Re: vmstat: On demand vmstat workers V8

2014-07-11 Thread Frederic Weisbecker
On Fri, Jul 11, 2014 at 10:17:41AM -0500, Christoph Lameter wrote: > On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > > > > Converted what? We still need to keep a cpumask around that tells us which > > > processor have vmstat running and which do not. > > > > > > > Converted to cpumask_var_t. >

Re: vmstat: On demand vmstat workers V8

2014-07-11 Thread Christoph Lameter
On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > > Converted what? We still need to keep a cpumask around that tells us which > > processor have vmstat running and which do not. > > > > Converted to cpumask_var_t. > > I mean we spent dozens emails on that... Oh there is this outstanding fix, ri

Re: vmstat: On demand vmstat workers V8

2014-07-11 Thread Frederic Weisbecker
On Fri, Jul 11, 2014 at 08:56:04AM -0500, Christoph Lameter wrote: > On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > > > > @@ -1228,20 +1244,105 @@ static const struct file_operations proc > > > #ifdef CONFIG_SMP > > > static DEFINE_PER_CPU(struct delayed_work, vmstat_work); > > > int sysctl_

Re: vmstat: On demand vmstat workers V8

2014-07-11 Thread Christoph Lameter
On Fri, 11 Jul 2014, Frederic Weisbecker wrote: > > @@ -1228,20 +1244,105 @@ static const struct file_operations proc > > #ifdef CONFIG_SMP > > static DEFINE_PER_CPU(struct delayed_work, vmstat_work); > > int sysctl_stat_interval __read_mostly = HZ; > > +struct cpumask *cpu_stat_off; > > I thou

Re: vmstat: On demand vmstat workers V8

2014-07-11 Thread Frederic Weisbecker
On Thu, Jul 10, 2014 at 09:04:55AM -0500, Christoph Lameter wrote: > > V7->V8 > - hackbench regression test shows a tiny performance increase due > to reduced OS processing. > - Rediff against 3.16-rc4. > > V6->V7 > - Remove /sysfs support. > > V5->V6: > - Shepherd thread as a general worker t