[PATCH -next 5/6] rcu: Document concurrent quiescent state reporting for offline CPUs

2025-07-15 Thread Joel Fernandes
@paulmck-laptop/ Co-developed-by: Paul E. McKenney Reviewed-by: Frederic Weisbecker Signed-off-by: Joel Fernandes --- .../RCU/Design/Requirements/Requirements.rst | 87 +++ kernel/rcu/tree.c | 19 +++- 2 files changed, 105 insertions(+), 1 deletion(-) diff

[PATCH -next 4/6] rcu: Document separation of rcu_state and rnp's gp_seq

2025-07-15 Thread Joel Fernandes
The details of this are subtle and was discussed recently. Add a quick-quiz about this and refer to it from the code, for more clarity. Reviewed-by: Paul E. McKenney Signed-off-by: Joel Fernandes --- .../Data-Structures/Data-Structures.rst | 32 +++ kernel/rcu/tree.c

[PATCH -next 3/6] rcu: Document GP init vs hotplug-scan ordering requirements

2025-07-15 Thread Joel Fernandes
Add detailed comments explaining the critical ordering constraints during RCU grace period initialization, based on discussions with Frederic. Reviewed-by: Paul E. McKenney Co-developed-by: Frederic Weisbecker Signed-off-by: Joel Fernandes --- .../RCU/Design/Requirements/Requirements.rst

[PATCH -rcu -next 7/7] rcu: Document concurrent quiescent state reporting for offline CPUs

2025-07-08 Thread Joel Fernandes
@paulmck-laptop/ Co-developed-by: Paul E. McKenney Reviewed-by: Frederic Weisbecker Signed-off-by: Joel Fernandes --- .../RCU/Design/Requirements/Requirements.rst | 87 +++ kernel/rcu/tree.c | 19 +++- 2 files changed, 105 insertions(+), 1 deletion(-) diff

[PATCH -rcu -next 5/7] rcu: Document GP init vs hotplug-scan ordering requirements

2025-07-08 Thread Joel Fernandes
Add detailed comments explaining the critical ordering constraints during RCU grace period initialization, based on discussions with Frederic. Co-developed-by: Frederic Weisbecker Signed-off-by: Joel Fernandes --- .../RCU/Design/Requirements/Requirements.rst | 41 +++ kernel

[PATCH -rcu -next 6/7] rcu: Document separation of rcu_state and rnp's gp_seq

2025-07-08 Thread Joel Fernandes
The details of this are subtle and was discussed recently. Add a quick-quiz about this and refer to it from the code, for more clarity. Signed-off-by: Joel Fernandes --- .../Data-Structures/Data-Structures.rst | 32 +++ kernel/rcu/tree.c | 4

Re: [PATCH] rcu: Document concurrent quiescent state reporting for offline CPUs

2025-07-07 Thread Joel Fernandes
On 7/7/2025 10:22 AM, Frederic Weisbecker wrote: > Le Sun, Jul 06, 2025 at 11:32:08PM -0400, Joel Fernandes a écrit : >> The synchronization of CPU offlining with GP initialization is confusing >> to put it mildly (rightfully so as the issue it deals with is complex). >&g

[PATCH] rcu: Document concurrent quiescent state reporting for offline CPUs

2025-07-06 Thread Joel Fernandes
@paulmck-laptop/ Co-developed-by: Paul E. McKenney Signed-off-by: Joel Fernandes --- .../RCU/Design/Requirements/Requirements.rst | 87 +++ kernel/rcu/tree.c | 19 +++- 2 files changed, 105 insertions(+), 1 deletion(-) diff --git a/Documentation/RCU/Design

[PATCH 2/2] rcu: Document separation of rcu_state and rnp's gp_seq

2025-07-06 Thread Joel Fernandes
The details of this are subtle and was discussed recently. Add a quick-quiz about this and refer to it from the code, for more clarity. Signed-off-by: Joel Fernandes --- .../Data-Structures/Data-Structures.rst | 32 +++ kernel/rcu/tree.c | 4

[PATCH 1/2] rcu: Document GP initialization vs hotplug ordering requirements

2025-07-06 Thread Joel Fernandes
Add detailed comments explaining the critical ordering constraints during RCU grace period initialization, based on discussions with Frederic. Co-developed-by: Frederic Weisbecker Signed-off-by: Joel Fernandes --- .../RCU/Design/Requirements/Requirements.rst | 41 +++ kernel

Re: [PATCH v5] rcutorture: Perform more frequent testing of ->gpwrap

2025-04-21 Thread Joel Fernandes
On 4/21/2025 2:32 PM, Paul E. McKenney wrote: > On Mon, Apr 21, 2025 at 01:53:17PM -0400, Joel Fernandes wrote: >> On 4/21/2025 1:47 PM, Joel Fernandes wrote: >>> Currently, the ->gpwrap is not tested (at all per my testing) due to the >>> requirement of a l

Re: [PATCH v5] rcutorture: Perform more frequent testing of ->gpwrap

2025-04-21 Thread Joel Fernandes
On 4/21/2025 1:47 PM, Joel Fernandes wrote: > Currently, the ->gpwrap is not tested (at all per my testing) due to the > requirement of a large delta between a CPU's rdp->gp_seq and its node's > rnp->gpseq. > > This results in no testing of ->gpwrap bein

[PATCH v5] rcutorture: Perform more frequent testing of ->gpwrap

2025-04-21 Thread Joel Fernandes
hen for 5 minutes, we switch to a smaller delta causing 1-2 wraps in 5 minutes. I believe this is reasonable since we at least add a little bit of testing for usecases where ->gpwrap is set. Tested-by: Paul E. McKenney Signed-off-by: Joel Fernandes --- v4->v5 - Added gpwrap_lag

[PATCH 6/7] doc: Update LWN RCU API links in whatisRCU.rst

2025-04-18 Thread Joel Fernandes
From: "Paul E. McKenney" This commit adds the 2024 LWN RCU API article set. Signed-off-by: Paul E. McKenney Signed-off-by: Joel Fernandes --- Documentation/RCU/whatisRCU.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Documentation/RCU/whatisRCU.rst b/Documen

[PATCH 7/7] doc/RCU/listRCU: refine example code for eliminating stale data

2025-04-18 Thread Joel Fernandes
https://lore.kernel.org/r/20250218005047.27258-1-richard.weiy...@gmail.com Signed-off-by: Boqun Feng Signed-off-by: Joel Fernandes --- Documentation/RCU/listRCU.rst | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/Documentation/RCU/listRCU.rst b/Documentation/RCU/listRCU.rst

Re: [PATCH v4] tracing: Support to dump instance traces by ftrace_dump_on_oops

2024-02-08 Thread Joel Fernandes
On 2/8/2024 4:22 AM, Huang Yiwei wrote: > > > On 2/7/2024 10:13 PM, Joel Fernandes wrote: [..] >>> Perhaps we need an option for these too, and have all options be >>> updated via the command line. That way we don't need to make special >>> boot line

Re: [PATCH v4] tracing: Support to dump instance traces by ftrace_dump_on_oops

2024-02-07 Thread Joel Fernandes
On Wed, Feb 7, 2024 at 6:14 AM Steven Rostedt wrote: > > On Wed, 7 Feb 2024 05:24:58 -0500 > Joel Fernandes wrote: > > > Btw, hopefully the "trace off on warning" and related boot parameters also > > apply > > to instances, I haven't personally check

Re: [PATCH v4] tracing: Support to dump instance traces by ftrace_dump_on_oops

2024-02-07 Thread Joel Fernandes
ffer and multiple instance buffer on all CPUs, or only dump on CPU > that triggered the oops if =2 or =orig_cpu is given > > Also, the sysctl node can handle the input accordingly. > > Cc: Ross Zwisler > Signed-off-by: Joel Fernandes (Google) > Signed-

Re: [PATCH 1/5] rcu/rcuperf: Add kfree_rcu() performance Tests

2019-09-03 Thread Joel Fernandes
On Tue, Sep 03, 2019 at 01:08:49PM -0700, Paul E. McKenney wrote: > On Thu, Aug 29, 2019 at 04:56:37PM -0400, Joel Fernandes wrote: > > On Wed, Aug 28, 2019 at 02:12:26PM -0700, Paul E. McKenney wrote: > > [ . . . ] > > > > > +static int > &

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-29 Thread Joel Fernandes
On Thu, Aug 29, 2019 at 05:47:56PM -0700, Paul E. McKenney wrote: [snip] > > > > Paul, also what what happens in the following scenario: > > > > > > > > CPU0 CPU1 > > > > > > > > A syscall causes rcu_eqs_exit() > > > > rcu_read_lock(); > > > >

Re: [PATCH 5/5] rcu: Remove kfree_call_rcu_nobatch()

2019-08-29 Thread Joel Fernandes
Hi Paul, I think this is the only contentious patch preventing my resend of the series, let me know what you think, I replied below: On Wed, Aug 28, 2019 at 02:56:36PM -0700, Paul E. McKenney wrote: > On Tue, Aug 27, 2019 at 03:01:59PM -0400, Joel Fernandes (Google) wrote: [snip] > > d

Re: [PATCH 4/5] rcu: Remove kfree_rcu() special casing and lazy handling

2019-08-29 Thread Joel Fernandes
All comments addressed on this one, thanks! - Joel On Wed, Aug 28, 2019 at 02:46:29PM -0700, Paul E. McKenney wrote: > On Tue, Aug 27, 2019 at 03:01:58PM -0400, Joel Fernandes (Google) wrote: > > Remove kfree_rcu() special casing and lazy handling from RCU. > > For Tiny

Re: [PATCH v2] rcu/tree: Add multiple in-flight batches of kfree_rcu work

2019-08-29 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 01:45:21PM -0700, Paul E. McKenney wrote: > On Wed, Aug 28, 2019 at 10:09:52AM -0400, Joel Fernandes (Google) wrote: > > During testing, it was observed that amount of memory consumed due > > kfree_rcu() batching is 300-400MB. Previously we had only a singl

Re: [PATCH 1/5] rcu/rcuperf: Add kfree_rcu() performance Tests

2019-08-29 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 02:12:26PM -0700, Paul E. McKenney wrote: > On Tue, Aug 27, 2019 at 03:01:55PM -0400, Joel Fernandes (Google) wrote: > > This test runs kfree_rcu() in a loop to measure performance of the new > > kfree_rcu() batching functionality. > > > > The fo

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-29 Thread Joel Fernandes
Hi Paul, On Thu, Aug 29, 2019 at 09:54:07AM -0700, Paul E. McKenney wrote: > On Thu, Aug 29, 2019 at 09:21:46AM -0700, Andy Lutomirski wrote: > > On Thu, Aug 29, 2019 at 9:10 AM Paul E. McKenney wrote: > > > > > > On Thu, Aug 29, 2019 at 10:43:55AM

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-29 Thread Joel Fernandes
On Thu, Aug 29, 2019 at 09:13:01AM -0700, Paul E. McKenney wrote: > On Thu, Aug 29, 2019 at 11:13:25AM -0400, Joel Fernandes wrote: > > On Thu, Aug 29, 2019 at 10:43:55AM -0400, Joel Fernandes wrote: > > > On Wed, Aug 28, 2019 at 08:43:36PM -0700, Paul E. McKenney

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-29 Thread Joel Fernandes
On Thu, Aug 29, 2019 at 10:43:55AM -0400, Joel Fernandes wrote: > On Wed, Aug 28, 2019 at 08:43:36PM -0700, Paul E. McKenney wrote: > [snip] > > > > > > This change is not fixing a bug, so there is no need for an > > > > > > emergency fix, > > &

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-29 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 08:43:36PM -0700, Paul E. McKenney wrote: [snip] > > > > > This change is not fixing a bug, so there is no need for an emergency > > > > > fix, > > > > > and thus no point in additional churn. I understand that it is a bit > > > > > annoying to code and test something and

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-29 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 08:43:36PM -0700, Paul E. McKenney wrote: [snip] > On the tracing patch... That patch might be a good idea regardless, > but I bet that the reason that you felt the sudden need for it was due > to the loss of information in your eventual ->dynticks_nesting field. > After a

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 04:12:47PM -0700, Paul E. McKenney wrote: > On Wed, Aug 28, 2019 at 06:14:44PM -0400, Joel Fernandes wrote: > > On Wed, Aug 28, 2019 at 03:01:08PM -0700, Paul E. McKenney wrote: > > > On Wed, Aug 28, 2019 at 05:42:41PM -0400, Joel Fernandes wrote: >

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 03:01:08PM -0700, Paul E. McKenney wrote: > On Wed, Aug 28, 2019 at 05:42:41PM -0400, Joel Fernandes wrote: > > On Wed, Aug 28, 2019 at 02:19:04PM -0700, Paul E. McKenney wrote: > > > On Wed, Aug 28, 2019 at 05:05:25PM -0400, Joel Fernandes wrote: >

Re: [RFC v1 1/2] rcu/tree: Clean up dynticks counter usage

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 01:13:44PM -0700, Paul E. McKenney wrote: > On Mon, Aug 26, 2019 at 09:33:53PM -0400, Joel Fernandes (Google) wrote: > > The dynticks counter are confusing due to crowbar writes of > > DYNTICK_IRQ_NONIDLE whose purpose is to detect half-interrupts (

Re: [PATCH 3/5] rcu/tree: Add support for debug_objects debugging for kfree_rcu()

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 02:31:19PM -0700, Paul E. McKenney wrote: > On Tue, Aug 27, 2019 at 03:01:57PM -0400, Joel Fernandes (Google) wrote: > > Make use of RCU's debug_objects debugging support > > (CONFIG_DEBUG_OBJECTS_RCU_HEAD) similar to call_rcu() and other flavors. >

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 02:19:04PM -0700, Paul E. McKenney wrote: > On Wed, Aug 28, 2019 at 05:05:25PM -0400, Joel Fernandes wrote: > > On Wed, Aug 28, 2019 at 01:23:30PM -0700, Paul E. McKenney wrote: > > > On Mon, Aug 26, 2019 at 09:33:54PM -0400, Joel Fernandes (Google)

Re: [PATCH 0/5] kfree_rcu() additions for -rcu

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 01:46:24PM -0700, Paul E. McKenney wrote: > On Wed, Aug 28, 2019 at 04:34:58PM -0400, Joel Fernandes wrote: > > On Wed, Aug 28, 2019 at 01:28:08PM -0700, Paul E. McKenney wrote: > > > On Tue, Aug 27, 2019 at 03:01:54PM -0400, Joel Fernandes (Google)

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 01:23:30PM -0700, Paul E. McKenney wrote: > On Mon, Aug 26, 2019 at 09:33:54PM -0400, Joel Fernandes (Google) wrote: > > The dynticks_nmi_nesting counter serves 4 purposes: > > > > (a) rcu_is_cpu_rrupt_from_idle() needs to be

Re: [PATCH 0/5] kfree_rcu() additions for -rcu

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 01:28:08PM -0700, Paul E. McKenney wrote: > On Tue, Aug 27, 2019 at 03:01:54PM -0400, Joel Fernandes (Google) wrote: > > Hi, > > > > This is a series on top of the patch "rcu/tree: Add basic support for > > kfree_rcu() batching". &g

[PATCH] rcu/dyntick-idle: Add better tracing

2019-08-28 Thread Joel Fernandes (Google)
evious patches to simplify rcu_dyntick counters [1] and with these traces, I have verified the counters are working properly. [1] Link: https://lore.kernel.org/patchwork/patch/1120021/ Link: https://lore.kernel.org/patchwork/patch/1120022/ Signed-off-by: Joel Fernandes (Google) --- incl

[PATCH v2] rcu/tree: Add multiple in-flight batches of kfree_rcu work

2019-08-28 Thread Joel Fernandes (Google)
more than 1 additional list did not show any improvement. Suggested-by: Paul E. McKenney Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/tree.c | 61 --- 1 file changed, 42 insertions(+), 19 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel

Re: [PATCH 2/5] rcu/tree: Add multiple in-flight batches of kfree_rcu work

2019-08-28 Thread Joel Fernandes
On Wed, Aug 28, 2019 at 07:52:53AM +0800, Boqun Feng wrote: > Hi Joel, > > On Tue, Aug 27, 2019 at 03:01:56PM -0400, Joel Fernandes (Google) wrote: > > During testing, it was observed that amount of memory consumed due > > kfree_rcu() batching is 300-400MB. Previously

[PATCH 3/5] rcu/tree: Add support for debug_objects debugging for kfree_rcu()

2019-08-27 Thread Joel Fernandes (Google)
e_rcu() calls. Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/tree.c | 8 1 file changed, 8 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 9b9ae4db1c2d..64568f12641d 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2757,6 +2757,7 @@ static

[PATCH 4/5] rcu: Remove kfree_rcu() special casing and lazy handling

2019-08-27 Thread Joel Fernandes (Google)
Remove kfree_rcu() special casing and lazy handling from RCU. For Tiny RCU we fold the special handling into just Tiny RCU code. Suggested-by: Paul E. McKenney Signed-off-by: Joel Fernandes (Google) --- Documentation/RCU/stallwarn.txt | 13 +++-- include/linux/rcu_segcblist.h | 2

[PATCH 0/5] kfree_rcu() additions for -rcu

2019-08-27 Thread Joel Fernandes (Google)
ree_rcu() is handled separately from call_rcu(), we also get rid of kfree "lazy" handling from tree RCU as suggested by Paul which will be unused. This also results in a nice negative delta as well. Joel Fernandes (Google) (5): rcu/rcuperf: Add kfree_rcu() performance Tests rcu/tree: Add multiple

[PATCH 5/5] rcu: Remove kfree_call_rcu_nobatch()

2019-08-27 Thread Joel Fernandes (Google)
Now that kfree_rcu() special casing have been removed from tree RCU, remove kfree_call_rcu_nobatch() since it is not needed. Signed-off-by: Joel Fernandes (Google) --- .../admin-guide/kernel-parameters.txt | 4 --- include/linux/rcutiny.h | 5 --- include/linux

[PATCH 1/5] rcu/rcuperf: Add kfree_rcu() performance Tests

2019-08-27 Thread Joel Fernandes (Google)
CONFIG_DEBUG_PREEMPT and CONFIG_PROVE_RCU for realistic comparisons with/without batching. Signed-off-by: Joel Fernandes (Google) --- .../admin-guide/kernel-parameters.txt | 17 ++ kernel/rcu/rcuperf.c | 181 +- 2 files changed, 190 insertions(+), 8

[PATCH 2/5] rcu/tree: Add multiple in-flight batches of kfree_rcu work

2019-08-27 Thread Joel Fernandes (Google)
more than 1 additional list did not show any improvement. Suggested-by: Paul E. McKenney Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/tree.c | 64 +-- 1 file changed, 45 insertions(+), 19 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel

Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-26 Thread Joel Fernandes
On Mon, Aug 26, 2019 at 9:34 PM Joel Fernandes (Google) wrote: > > The dynticks_nmi_nesting counter serves 4 purposes: > And actually, I meant 3 purposes ;-) :-P thanks, - Joel > (a) rcu_is_cpu_rrupt_from_idle() needs to be able to detect first > interrup

[RFC v1 1/2] rcu/tree: Clean up dynticks counter usage

2019-08-26 Thread Joel Fernandes (Google)
also do cheaper comparisons with zero instead for the code that keeps the tick on in rcu_nmi_enter_common(). In the next patch, both of the concerns of (2) will be addressed and then we can get rid of dynticks_nmi_nesting, however one step at a time. Signed-off-by: Joel Fernandes (Google) --- kernel/

[RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter

2019-08-26 Thread Joel Fernandes (Google)
_tick would have been set to true in the outermost interrupt, so the nested/NMI interrupts will check forced_tick anyway, and bail. Signed-off-by: Joel Fernandes (Google) --- .../Data-Structures/Data-Structures.rst | 31 +++-- Documentation/RCU/stallwarn.txt | 6 +- kerne

[RFC v1 0/2] RCU dyntick nesting counter cleanups

2019-08-26 Thread Joel Fernandes (Google)
simple counter and can be "crowbarred" in common situations. Several nights of rcutorture testing with CONFIG_RCU_EQS_DEBUG on all RCU kernel configurations have survived without any splats. Further testing is in progress, hence marked as RFC! thanks, - Joel Joel Fernandes (Google) (2)

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-19 Thread Joel Fernandes
On Wed, Aug 14, 2019 at 09:56:01AM +0200, Michal Hocko wrote: [snip] > > > > Can this be used to observe which library pages other processes are > > > > accessing, even if you don't have access to those processes, as long > > > > as you can map the same libraries? I realize that there are already a

Re: [PATCH v3 -rcu] workqueue: Convert for_each_wq to use built-in list check

2019-08-15 Thread Joel Fernandes
On Thu, Aug 15, 2019 at 07:57:49AM -0700, Matthew Wilcox wrote: > On Thu, Aug 15, 2019 at 10:18:42AM -0400, Joel Fernandes (Google) wrote: > > list_for_each_entry_rcu now has support to check for RCU reader sections > > as well as lock. Just use the support in it, instea

[PATCH v3 -rcu] workqueue: Convert for_each_wq to use built-in list check

2019-08-15 Thread Joel Fernandes (Google)
list_for_each_entry_rcu now has support to check for RCU reader sections as well as lock. Just use the support in it, instead of explicitly checking in the caller. Acked-by: Tejun Heo Signed-off-by: Joel Fernandes (Google) --- v1->v3: Changed lock_is_held() to lockdep_is_held() ker

Re: [PATCH 1/3] workqueue: Convert for_each_wq to use built-in list check (v2)

2019-08-14 Thread Joel Fernandes
On Wed, Aug 14, 2019 at 12:48:41PM -0700, Tejun Heo wrote: > Hello, Joel. > > On Sun, Aug 11, 2019 at 06:11:09PM -0400, Joel Fernandes (Google) wrote: > > list_for_each_entry_rcu now has support to check for RCU reader sections > > as well as lock. Just use the supp

Re: [PATCH v3 1/2] rcu/tree: Add basic support for kfree_rcu batching

2019-08-14 Thread Joel Fernandes
On Wed, Aug 14, 2019 at 11:44:29AM -0700, Paul E. McKenney wrote: > On Wed, Aug 14, 2019 at 01:22:33PM -0400, Joel Fernandes wrote: > > On Wed, Aug 14, 2019 at 10:38:17AM -0400, Joel Fernandes wrote: > > > On Tue, Aug 13, 2019 at 12:07:38PM -0700, Paul E. McKenney

Re: [PATCH v3 1/2] rcu/tree: Add basic support for kfree_rcu batching

2019-08-14 Thread Joel Fernandes
On Wed, Aug 14, 2019 at 10:38:17AM -0400, Joel Fernandes wrote: > On Tue, Aug 13, 2019 at 12:07:38PM -0700, Paul E. McKenney wrote: [snip] > > > - * Queue an RCU callback for lazy invocation after a grace period. > > > - * This will likely be later named somethin

Re: [PATCH v5 2/6] mm/page_idle: Add support for handling swapped PG_Idle pages

2019-08-14 Thread Joel Fernandes
On Wed, Aug 14, 2019 at 10:05:31AM +0200, Michal Hocko wrote: > On Tue 13-08-19 11:36:59, Joel Fernandes wrote: > > On Tue, Aug 13, 2019 at 05:04:50PM +0200, Michal Hocko wrote: > > > On Wed 07-08-19 13:15:55, Joel Fernandes (Google) wrote: > > > > Idle page trackin

Re: [PATCH v3 1/2] rcu/tree: Add basic support for kfree_rcu batching

2019-08-14 Thread Joel Fernandes
[1] > > http://lore.kernel.org/lkml/20190723035725-mutt-send-email-...@kernel.org > > [2] https://lkml.org/lkml/2017/12/19/824 > > > > Cc: Rao Shoaib > > Cc: max.byungchul.p...@gmail.com > > Cc: byungchul.p...@lge.com > > Cc: kernel-t...@an

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-13 Thread Joel Fernandes
On Tue, Aug 13, 2019 at 05:34:16PM +0200, Daniel Gruss wrote: > On 8/13/19 5:29 PM, Jann Horn wrote: > > On Tue, Aug 13, 2019 at 12:09 PM Michal Hocko wrote: > >> On Mon 12-08-19 20:14:38, Jann Horn wrote: > >>> On Wed, Aug 7, 2019 at 7:16 PM Joel Fernandes (Go

[PATCH v3 2/2] rcuperf: Add kfree_rcu performance Tests

2019-08-13 Thread Joel Fernandes (Google)
CONFIG_PROVE_RCU for realistic comparisons with/without batching. Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/rcuperf.c | 189 +-- 1 file changed, 181 insertions(+), 8 deletions(-) diff --git a/kernel/rcu/rcuperf.c b/kernel/rcu/rcuperf.c index

[PATCH v3 1/2] rcu/tree: Add basic support for kfree_rcu batching

2019-08-13 Thread Joel Fernandes (Google)
chul.p...@lge.com Cc: kernel-t...@android.com Cc: kernel-t...@lge.com Co-developed-by: Byungchul Park Signed-off-by: Byungchul Park Signed-off-by: Joel Fernandes (Google) --- v2->v3: Just some code comment changes thanks to Byungchul. RFCv1->PATCH v2: Removed limits on the -&g

Re: [PATCH v5 2/6] mm/page_idle: Add support for handling swapped PG_Idle pages

2019-08-13 Thread Joel Fernandes
On Tue, Aug 13, 2019 at 05:04:50PM +0200, Michal Hocko wrote: > On Wed 07-08-19 13:15:55, Joel Fernandes (Google) wrote: > > Idle page tracking currently does not work well in the following > > scenario: > > 1. mark page-A idle which was present at that time. > > 2.

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-13 Thread Joel Fernandes
On Mon, Aug 12, 2019 at 08:14:38PM +0200, Jann Horn wrote: [snip] > > +/* Helper to get the start and end frame given a pos and count */ > > +static int page_idle_get_frames(loff_t pos, size_t count, struct mm_struct > > *mm, > > + unsigned long *start, unsigned long

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-13 Thread Joel Fernandes
On Tue, Aug 13, 2019 at 04:14:32PM +0200, Michal Hocko wrote: [snip] > > > If the API is flawed then this is likely going > > > to kick us later and will be hard to fix. I am still not convinced about > > > the swap part of the thing TBH. > > > > Ok, then let us discuss it. As I mentioned before,

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-13 Thread Joel Fernandes
On Tue, Aug 13, 2019 at 12:08:56PM +0200, Michal Hocko wrote: > On Mon 12-08-19 20:14:38, Jann Horn wrote: > > On Wed, Aug 7, 2019 at 7:16 PM Joel Fernandes (Google) > > wrote: > > > The page_idle tracking feature currently requires looking up the pagemap > &g

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-13 Thread Joel Fernandes
On Tue, Aug 13, 2019 at 11:14:30AM +0200, Michal Hocko wrote: > On Mon 12-08-19 10:56:20, Joel Fernandes wrote: > > On Thu, Aug 08, 2019 at 10:00:44AM +0200, Michal Hocko wrote: > > > On Wed 07-08-19 17:31:05, Joel Fernandes wrote: > > > > On Wed, Aug 07, 2019 at 0

Re: [PATCH v2] driver/core: Fix build error when SRCU and lockdep disabled

2019-08-13 Thread Joel Fernandes
On Tue, Aug 13, 2019 at 09:39:05AM -0400, Joel Fernandes wrote: [snip] > > > drivers/base/core.c | 4 +++- > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/base/core.c b/drivers/base/core.c > > > index 32cf83d1

Re: [PATCH v2] driver/core: Fix build error when SRCU and lockdep disabled

2019-08-13 Thread Joel Fernandes
On Tue, Aug 13, 2019 at 08:05:40AM +0200, Greg Kroah-Hartman wrote: > On Mon, Aug 12, 2019 at 05:49:17PM -0400, Joel Fernandes (Google) wrote: > > Check if lockdep lock checking is disabled. If so, then do not define > > device_links_read_lock_held(). It is used only from place

[PATCH v2] driver/core: Fix build error when SRCU and lockdep disabled

2019-08-12 Thread Joel Fernandes (Google)
: Lai Jiangshan , Cc: linux-doc@vger.kernel.org, Cc: Mathieu Desnoyers , Cc: "Paul E. McKenney" , Cc: "Rafael J. Wysocki" , Cc: r...@vger.kernel.org, Cc: Steven Rostedt , Reported-by: kbuild test robot Signed-off-by: Joel Fernandes (Google) --- drivers/base/core.c | 4 +++-

Re: [PATCH 2/3] doc: Update documentation about list_for_each_entry_rcu (v1)

2019-08-12 Thread Joel Fernandes
On Mon, Aug 12, 2019 at 01:22:41PM -0700, Paul E. McKenney wrote: > On Sun, Aug 11, 2019 at 06:11:10PM -0400, Joel Fernandes (Google) wrote: > > This patch updates the documentation with information about > > usage of lockdep with list_for_each_entry_rcu(). > > > > Si

Re: [PATCH 3/3] driver/core: Fix build error when SRCU and lockdep disabled

2019-08-12 Thread Joel Fernandes
On Mon, Aug 12, 2019 at 02:11:19PM -0400, Steven Rostedt wrote: > On Mon, 12 Aug 2019 09:03:10 -0400 > Joel Fernandes wrote: > > > > > > drivers/base/core.c | 6 +- > > > > 1 file changed, 5 insertions(+), 1 deletion(-) > > > > > &g

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-12 Thread Joel Fernandes
On Thu, Aug 08, 2019 at 10:00:44AM +0200, Michal Hocko wrote: > On Wed 07-08-19 17:31:05, Joel Fernandes wrote: > > On Wed, Aug 07, 2019 at 01:58:40PM -0700, Andrew Morton wrote: > > > On Wed, 7 Aug 2019 16:45:30 -0400 Joel Fernandes > > > wrote: > > > > &

Re: [PATCH v2 1/2] rcu/tree: Add basic support for kfree_rcu batching

2019-08-12 Thread Joel Fernandes
On Mon, Aug 12, 2019 at 07:29:17PM +0900, Byungchul Park wrote: [snip] > > diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h > > index 8e727f57d814..383f2481750f 100644 > > --- a/include/linux/rcutiny.h > > +++ b/include/linux/rcutiny.h > > @@ -39,6 +39,11 @@ static inline void kfree_c

Re: [PATCH 3/3] driver/core: Fix build error when SRCU and lockdep disabled

2019-08-12 Thread Joel Fernandes
On Mon, Aug 12, 2019 at 07:02:56AM +0200, Greg Kroah-Hartman wrote: > On Sun, Aug 11, 2019 at 06:11:11PM -0400, Joel Fernandes (Google) wrote: > > Properly check if lockdep lock checking is disabled at config time. If > > so, then lock_is_held() is undefined so don't do any che

Re: [PATCH 1/3] workqueue: Convert for_each_wq to use built-in list check (v2)

2019-08-11 Thread Joel Fernandes
On Sun, Aug 11, 2019 at 6:11 PM Joel Fernandes (Google) wrote: > > list_for_each_entry_rcu now has support to check for RCU reader sections > as well as lock. Just use the support in it, instead of explicitly > checking in the caller. > > Signed-off-by: Joel Fernandes (Google)

[PATCH 2/3] doc: Update documentation about list_for_each_entry_rcu (v1)

2019-08-11 Thread Joel Fernandes (Google)
This patch updates the documentation with information about usage of lockdep with list_for_each_entry_rcu(). Signed-off-by: Joel Fernandes (Google) --- Documentation/RCU/lockdep.txt | 15 +++ Documentation/RCU/whatisRCU.txt | 9 - 2 files changed, 19 insertions(+), 5

[PATCH 1/3] workqueue: Convert for_each_wq to use built-in list check (v2)

2019-08-11 Thread Joel Fernandes (Google)
list_for_each_entry_rcu now has support to check for RCU reader sections as well as lock. Just use the support in it, instead of explicitly checking in the caller. Signed-off-by: Joel Fernandes (Google) --- kernel/workqueue.c | 10 ++ 1 file changed, 2 insertions(+), 8 deletions

[PATCH 3/3] driver/core: Fix build error when SRCU and lockdep disabled

2019-08-11 Thread Joel Fernandes (Google)
2fee8 ("acpi: Use built-in RCU list checking for acpi_ioremaps list") Reported-by: kbuild test robot Signed-off-by: Joel Fernandes (Google) --- This patch is based on the -rcu dev branch. drivers/base/core.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/driver

[PATCH v2 2/2] rcuperf: Add kfree_rcu performance Tests

2019-08-10 Thread Joel Fernandes (Google)
CONFIG_PROVE_RCU for realistic comparisons with/without batching. Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/rcuperf.c | 189 +-- 1 file changed, 181 insertions(+), 8 deletions(-) diff --git a/kernel/rcu/rcuperf.c b/kernel/rcu/rcuperf.c index

[PATCH v2 1/2] rcu/tree: Add basic support for kfree_rcu batching

2019-08-10 Thread Joel Fernandes (Google)
5725-mutt-send-email-...@kernel.org [2] https://lkml.org/lkml/2017/12/19/824 Cc: Rao Shoaib Cc: max.byungchul.p...@gmail.com Cc: byungchul.p...@lge.com Cc: kernel-t...@android.com Cc: kernel-t...@lge.com Co-developed-by: Byungchul Park Signed-off-by: Byungchul Park Signed-off-by: Joel Fernandes (Google)

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-07 Thread Joel Fernandes
On Wed, Aug 07, 2019 at 05:31:05PM -0400, Joel Fernandes wrote: > On Wed, Aug 07, 2019 at 01:58:40PM -0700, Andrew Morton wrote: > > On Wed, 7 Aug 2019 16:45:30 -0400 Joel Fernandes > > wrote: > > > > > On Wed, Aug 07, 2019 at 01:04:02PM -0700, Andrew Morton wrot

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-07 Thread Joel Fernandes
On Wed, Aug 07, 2019 at 01:58:40PM -0700, Andrew Morton wrote: > On Wed, 7 Aug 2019 16:45:30 -0400 Joel Fernandes > wrote: > > > On Wed, Aug 07, 2019 at 01:04:02PM -0700, Andrew Morton wrote: > > > On Wed, 7 Aug 2019 13:15:54 -0400 "Joel Fernandes (Google)"

Re: [PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-07 Thread Joel Fernandes
On Wed, Aug 07, 2019 at 01:04:02PM -0700, Andrew Morton wrote: > On Wed, 7 Aug 2019 13:15:54 -0400 "Joel Fernandes (Google)" > wrote: > > > In Android, we are using this for the heap profiler (heapprofd) which > > profiles and pin points code paths which alloc

Re: [PATCH v4 1/5] mm/page_idle: Add per-pid idle page tracking using virtual indexing

2019-08-07 Thread Joel Fernandes
On Wed, Aug 07, 2019 at 01:01:22PM -0700, Andrew Morton wrote: > On Wed, 7 Aug 2019 06:00:13 -0400 Joel Fernandes > wrote: > > > > > 8 files changed, 376 insertions(+), 45 deletions(-) > > > > > > Quite a lot of new code unconditionally added to major

[PATCH v5 6/6] doc: Update documentation for page_idle virtual address indexing

2019-08-07 Thread Joel Fernandes (Google)
This patch updates the documentation with the new page_idle tracking feature which uses virtual address indexing. Reviewed-by: Mike Rapoport Reviewed-by: Sandeep Patil Signed-off-by: Joel Fernandes (Google) --- .../admin-guide/mm/idle_page_tracking.rst | 43 --- 1 file

[PATCH v5 1/6] mm/page_idle: Add per-pid idle page tracking using virtual index

2019-08-07 Thread Joel Fernandes (Google)
ff-by: Joel Fernandes (Google) --- v3->v4: Minor fixups (Minchan) Add swap pte handling (Konstantin, Minchan) v2->v3: Fixed a bug where I was doing a kfree that is not needed due to not needing to do GFP_ATOMIC allocations. v1->v2: Mark swap ptes as idle (Minchan) Avoid need

[PATCH v5 4/6] [RFC] arm64: Add support for idle bit in swap PTE

2019-08-07 Thread Joel Fernandes (Google)
page tracking only works on user pages in the LRU. Device pages should not consitute those so it should be unused and safe to use. Cc: Robin Murphy Signed-off-by: Joel Fernandes (Google) --- arch/arm64/Kconfig| 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64

[PATCH v5 3/6] [RFC] x86: Add support for idle bit in swap PTE

2019-08-07 Thread Joel Fernandes (Google)
swap PTE (see the comment in arch/x86/include/asm/pgtable_64.h). Bit 2 corresponds to _PAGE_USER. Use it for swap PTE purposes. Signed-off-by: Joel Fernandes (Google) --- arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 15 +++ arch/x86/include/asm

[PATCH v5 5/6] page_idle: Drain all LRU pagevec before idle tracking

2019-08-07 Thread Joel Fernandes (Google)
patch drains all CPU's pagevec before starting idle tracking. Signed-off-by: Joel Fernandes (Google) --- mm/page_idle.c | 21 + 1 file changed, 21 insertions(+) diff --git a/mm/page_idle.c b/mm/page_idle.c index 2766d4ab348c..26440a497609 100644 --- a/mm/page_idle.c

[PATCH v5 2/6] mm/page_idle: Add support for handling swapped PG_Idle pages

2019-08-07 Thread Joel Fernandes (Google)
oid's ActivityManager) to swap pages out of a process that it knows will be cold. To an external process like a heap profiler that is doing idle tracking on another process, this procedure will interfere with the idle page tracking similar to the above steps. Suggested-by: Minchan Kim Signed-of

Re: [PATCH v4 1/5] mm/page_idle: Add per-pid idle page tracking using virtual indexing

2019-08-07 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 03:19:21PM -0700, Andrew Morton wrote: > (cc Brendan's other email address, hoping for review input ;)) ;) > On Mon, 5 Aug 2019 13:04:47 -0400 "Joel Fernandes (Google)" > wrote: > > > The page_idle tracking feature currently requires

Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 11:47:47PM +0900, Minchan Kim wrote: > On Tue, Aug 06, 2019 at 01:57:03PM +0200, Michal Hocko wrote: > > On Tue 06-08-19 07:14:46, Joel Fernandes wrote: > > > On Tue, Aug 06, 2019 at 12:47:55PM +0200, Michal Hocko wrote: > > > > On Tue 06-

Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 01:44:02PM +0200, Michal Hocko wrote: [snip] > > > > This operation even if expensive is only done once during the access of > > > > the > > > > page_idle file. Did you have a better fix in mind? > > > > > > Can we set the idle bit also for non-lru pages as long as they ar

Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 01:57:03PM +0200, Michal Hocko wrote: > On Tue 06-08-19 07:14:46, Joel Fernandes wrote: > > On Tue, Aug 06, 2019 at 12:47:55PM +0200, Michal Hocko wrote: > > > On Tue 06-08-19 06:36:27, Joel Fernandes wrote: > > > > On Tue, Aug 06, 2019 at

Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 01:14:52PM +0200, Michal Hocko wrote: > On Tue 06-08-19 20:07:37, Minchan Kim wrote: > > On Tue, Aug 06, 2019 at 12:47:55PM +0200, Michal Hocko wrote: > > > On Tue 06-08-19 06:36:27, Joel Fernandes wrote: > > > > On Tue, Aug 06, 2019 at 10:42:

Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 12:51:49PM +0200, Michal Hocko wrote: > On Tue 06-08-19 06:45:54, Joel Fernandes wrote: > > On Tue, Aug 06, 2019 at 10:43:57AM +0200, Michal Hocko wrote: > > > On Mon 05-08-19 13:04:50, Joel Fernandes (Google) wrote: > > > > During idle t

Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 12:47:55PM +0200, Michal Hocko wrote: > On Tue 06-08-19 06:36:27, Joel Fernandes wrote: > > On Tue, Aug 06, 2019 at 10:42:03AM +0200, Michal Hocko wrote: > > > On Mon 05-08-19 13:04:49, Joel Fernandes (Google) wrote: > > > > This bit will be u

Re: [PATCH v4 1/5] mm/page_idle: Add per-pid idle page tracking using virtual indexing

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 10:56:05AM +0200, Michal Hocko wrote: > On Mon 05-08-19 13:04:47, Joel Fernandes (Google) wrote: > > The page_idle tracking feature currently requires looking up the pagemap > > for a process followed by interacting with /sys/kernel/mm/page_idle. > >

Re: [PATCH v4 4/5] page_idle: Drain all LRU pagevec before idle tracking

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 10:43:57AM +0200, Michal Hocko wrote: > On Mon 05-08-19 13:04:50, Joel Fernandes (Google) wrote: > > During idle tracking, we see that sometimes faulted anon pages are in > > pagevec but are not drained to LRU. Idle tracking considers pages only > > on

Re: [PATCH v4 3/5] [RFC] arm64: Add support for idle bit in swap PTE

2019-08-06 Thread Joel Fernandes
On Tue, Aug 06, 2019 at 10:42:03AM +0200, Michal Hocko wrote: > On Mon 05-08-19 13:04:49, Joel Fernandes (Google) wrote: > > This bit will be used by idle page tracking code to correctly identify > > if a page that was swapped out was idle before it got swapped out. > > Wit

  1   2   3   >