[PATCH tip/core/rcu 21/21] rcu: Do any deferred nocb wakeups at CPU offline time

2021-01-05 Thread paulmck
From: "Paul E. McKenney" Because the need to wake a nocb GP kthread ("rcuog") is sometimes detected when wakeups cannot be done, these wakeups can be deferred. The wakeups are then carried out by calls to do_nocb_deferred_wakeup() at various safe points in the code, including RCU's idle hooks. H

[PATCH tip/core/rcu 06/21] rcu/nocb: De-offloading GP kthread

2021-01-05 Thread paulmck
From: Frederic Weisbecker To de-offload callback processing back onto a CPU, it is necessary to clear SEGCBLIST_OFFLOAD and notify the nocb GP kthread, which will then clear its own bit flag and ignore this CPU until further notice. Whichever of the nocb CB and nocb GP kthreads is last to clear i

[PATCH tip/core/rcu 18/21] rcu/nocb: Add grace period and task state to show_rcu_nocb_state() output

2021-01-05 Thread paulmck
From: "Paul E. McKenney" This commit improves debuggability by indicating which grace period each batch of nocb callbacks is waiting on and by showing the task state and last CPU for reach nocb kthread. [ paulmck: Handle !SMP CB offloading per kernel test robot feedback. ] Signed-off-

[PATCH tip/core/rcu 09/21] rcu/nocb: Flush bypass before setting SEGCBLIST_SOFTIRQ_ONLY

2021-01-05 Thread paulmck
From: Frederic Weisbecker This commit flushes the bypass queue and sets state to avoid its being refilled before switching to the final de-offloaded state. To avoid refilling, this commit sets SEGCBLIST_SOFTIRQ_ONLY before re-enabling IRQs. Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Des

[PATCH tip/core/rcu 13/21] rcu/nocb: Locally accelerate callbacks as long as offloading isn't complete

2021-01-05 Thread paulmck
From: Frederic Weisbecker The local callbacks processing checks if any callbacks need acceleration. This commit carries out this checking under nocb lock protection in the middle of toggle operations, during which time rcu_core() executes concurrently with GP/CB kthreads. Cc: Josh Triplett Cc:

[PATCH tip/core/rcu 10/21] rcu/nocb: Set SEGCBLIST_SOFTIRQ_ONLY at the very last stage of de-offloading

2021-01-05 Thread paulmck
From: Frederic Weisbecker This commit sets SEGCBLIST_SOFTIRQ_ONLY once toggling is otherwise fully complete, allowing further RCU callback manipulation to be carried out locklessly and locally. Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Desnoyers Cc: Lai Jiangshan Cc: Joel Fernandes C

[PATCH tip/core/rcu 08/21] rcu/nocb: Shutdown nocb timer on de-offloading

2021-01-05 Thread paulmck
From: Frederic Weisbecker This commit ensures that the nocb timer is shut down before reaching the final de-offloaded state. The key goal is to prevent the timer handler from manipulating the callbacks without the protection of the nocb locks. Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu

[PATCH tip/core/rcu 1/3] rcu: Make RCU_BOOST default on CONFIG_PREEMPT_RT

2021-01-06 Thread paulmck
From: Sebastian Andrzej Siewior On PREEMPT_RT kernels, RCU callbacks are deferred to the `rcuc' kthread. This can stall RCU grace periods due to lengthy preemption not only of RCU readers but also of 'rcuc' kthreads, either of which prevent grace periods from completing, which can in turn result

[PATCH tip/core/rcu 3/3] rcu: Enable rcu_normal_after_boot unconditionally for RT

2021-01-06 Thread paulmck
disrupting real-time applications. Suggested-by: Luiz Capitulino Acked-by: Paul E. McKenney Signed-off-by: Julia Cartwright Signed-off-by: Sebastian Andrzej Siewior [ paulmck: Update kernel-parameters.txt accordingly. ] Signed-off-by: Paul E. McKenney --- Documentation/admin-guide/kernel

[PATCH tip/core/rcu 2/3] rcu: Unconditionally use rcuc threads on PREEMPT_RT

2021-01-06 Thread paulmck
Scott Wood [bigeasy: Reword commit message] Signed-off-by: Sebastian Andrzej Siewior [ paulmck: Update kernel-parameters.txt accordingly. ] Signed-off-by: Paul E. McKenney --- Documentation/admin-guide/kernel-parameters.txt | 4 kernel/rcu/tree.c | 4 +++- 2 files c

[PATCH tip/core/rcu 1/4] rcu: Mark obtuse portion of stall warning as internal debug

2021-01-06 Thread paulmck
From: "Paul E. McKenney" There is a rather obtuse string that can be printed as part of an expedited RCU CPU stall-warning message that starts with "blocking rcu_node structures". Under normal conditions, most of this message is just repeating the list of CPUs blocking the current expedited grac

[PATCH tip/core/rcu 3/4] rcu: Do not NMI offline CPUs

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Currently, RCU CPU stall warning messages will NMI whatever CPU looks like it is blocking either the current grace period or the grace-period kthread. This can produce confusing output if the target CPU is offline. This commit therefore checks for offline CPUs. Signed-o

[PATCH tip/core/rcu 2/4] rcu: For RCU grace-period kthread starvation, dump last CPU it ran on

2021-01-06 Thread paulmck
From: "Paul E. McKenney" When the RCU CPU stall-warning code detects that the RCU grace-period kthread is being starved, it dumps that kthread's stack. This can sometimes be useful, but it is also useful to know what is running on the CPU that this kthread is attempting to run on. This commit t

[PATCH tip/core/rcu 4/4] rcu: Check and report missed fqs timer wakeup on RCU stall

2021-01-06 Thread paulmck
From: Neeraj Upadhyay For a new grace period request, the RCU GP kthread transitions through following states: a. [RCU_GP_WAIT_GPS] -> [RCU_GP_DONE_GPS] The RCU_GP_WAIT_GPS state is where the GP kthread waits for a request for a new GP. Once it receives a request (for example, when a new RCU c

[PATCH tip/core/rcu 02/10] srcu: Provide internal interface to start a Tiny SRCU grace period

2021-01-06 Thread paulmck
From: "Paul E. McKenney" There is a need for a polling interface for SRCU grace periods. This polling needs to initiate an SRCU grace period without having to queue (and manage) a callback. This commit therefore splits the Tiny SRCU call_srcu() function into callback-queuing and start-grace-peri

[PATCH tip/core/rcu 04/10] srcu: Provide polling interfaces for Tiny SRCU grace periods

2021-01-06 Thread paulmck
value from either get_state_synchronize_srcu() or start_poll_synchronize_srcu() must be passed in to a later call to poll_state_synchronize_srcu(). Link: https://lore.kernel.org/rcu/20201112201547.gf3365...@moria.home.lan/ Reported-by: Kent Overstreet [ paulmck: Add EXPORT_SYMBOL_GPL() per kernel

[PATCH tip/core/rcu 01/10] srcu: Make Tiny SRCU use multi-bit grace-period counter

2021-01-06 Thread paulmck
an/ Reported-by: Kent Overstreet [ paulmck: Fix ->srcu_lock_nesting[] indexing per Neeraj Upadhyay. ] Reviewed-by: Neeraj Upadhyay Signed-off-by: Paul E. McKenney --- include/linux/srcutiny.h | 6 +++--- kernel/rcu/srcutiny.c| 5 +++-- 2 files changed, 6 insertions(+), 5 deletions(-) di

[PATCH tip/core/rcu 05/10] srcu: Provide polling interfaces for Tree SRCU grace periods

2021-01-06 Thread paulmck
value from either get_state_synchronize_srcu() or start_poll_synchronize_srcu() must be passed in to a later call to poll_state_synchronize_srcu(). Link: https://lore.kernel.org/rcu/20201112201547.gf3365...@moria.home.lan/ Reported-by: Kent Overstreet [ paulmck: Add EXPORT_SYMBOL_GPL() per kernel

[PATCH tip/core/rcu 06/10] srcu: Document polling interfaces for Tree SRCU grace periods

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds requirements documentation for the get_state_synchronize_srcu(), start_poll_synchronize_srcu(), and poll_state_synchronize_srcu() functions. Link: https://lore.kernel.org/rcu/20201112201547.gf3365...@moria.home.lan/ Reported-by: Kent Overstreet Reviewed

[PATCH tip/core/rcu 03/10] srcu: Provide internal interface to start a Tree SRCU grace period

2021-01-06 Thread paulmck
From: "Paul E. McKenney" There is a need for a polling interface for SRCU grace periods. This polling needs to initiate an SRCU grace period without having to queue (and manage) a callback. This commit therefore splits the Tree SRCU __call_srcu() function into callback-initialization and queuing

[PATCH tip/core/rcu 08/10] rcutorture: Prepare for ->start_gp_poll and ->poll_gp_state

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The new get_state_synchronize_srcu(), start_poll_synchronize_srcu() and poll_state_synchronize_srcu() functions need to be tested, and so this commit prepares by renaming the rcu_torture_ops field ->get_state to ->get_gp_state in order to be consistent with the upcoming -

[PATCH tip/core/rcu 07/10] srcu: Add comment explaining cookie overflow/wrap

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds to the poll_state_synchronize_srcu() header comment describing the issues surrounding SRCU cookie overflow/wrap for the different kernel configurations. Link: https://lore.kernel.org/rcu/20201112201547.gf3365...@moria.home.lan/ Reported-by: Kent Overstre

[PATCH tip/core/rcu 09/10] rcutorture: Add writer-side tests of polling grace-period API

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds writer-side testing of the polling grace-period API. One test verifies that the polling API sees a grace period caused by some other mechanism. Another test verifies that using the polling API to wait for a grace period does not result in too-short grace

[PATCH tip/core/rcu 10/10] rcutorture: Add reader-side tests of polling grace-period API

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds reader-side testing of the polling grace-period API. This testing verifies that a cookie obtained in an SRCU read-side critical section does not get a true return from poll_state_synchronize_srcu() within that same critical section. Link: https://lore.ke

[PATCH tip/core/rcu 14/17] torture: Clean up after torture-test CPU hotplugging

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit puts all CPUs back online at the end of a torture test, and also unconditionally puts them online at the beginning of the test, rather than just in the case of built-in tests. This allows torture tests to behave in a predictable manner, whether built-in or ba

[PATCH tip/core/rcu 16/17] torture: Break affinity of kthreads last running on outgoing CPU

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The advent of commit 06249738a41a ("workqueue: Manually break affinity on hotplug") means that the scheduler no longer silently breaks affinity for kthreads pinned to the outgoing CPU. This can happen for many of rcutorture's kthreads due to shuffling, which periodically

[PATCH tip/core/rcu 13/17] rcutorture: Make object_debug also double call_rcu() heap object

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit provides a test for call_rcu() printing the allocation address of a double-freed callback by double-freeing a callback allocated via kmalloc(). However, this commit does not depend on any other commit. Signed-off-by: Paul E. McKenney --- kernel/rcu/rcutort

[PATCH tip/core/rcu 09/17] torture: Make stutter use torture_hrtimeout_*() functions

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit saves a few lines of code by making the stutter_wait() and torture_stutter() functions use torture_hrtimeout_jiffies() and torture_hrtimeout_us(). Signed-off-by: Paul E. McKenney --- kernel/torture.c | 20 +--- 1 file changed, 5 insertions(+

[PATCH tip/core/rcu 17/17] rcutorture: Add rcutree.use_softirq=0 to RUDE01 and TASKS01

2021-01-06 Thread paulmck
From: "Paul E. McKenney" RCU's rcutree.use_softirq=0 kernel boot parameter substitutes the per-CPU rcuc kthreads for softirq, which is used in real-time installations. However, none of the rcutorture scenarios test this parameter. This commit therefore adds rcutree.use_softirq=0 to the RUDE01 and

[PATCH tip/core/rcu 10/17] rcutorture: Use hrtimers for reader and writer delays

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit replaces schedule_timeout_uninterruptible() and schedule_timeout_interruptible() with torture_hrtimeout_us() and torture_hrtimeout_jiffies() to avoid timer-wheel synchronization. Signed-off-by: Paul E. McKenney --- kernel/rcu/rcutorture.c | 7 +++ 1 fil

[PATCH tip/core/rcu 06/17] rcutorture: Make rcu_torture_fakewriter() use blocking wait primitives

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Full testing of the new SRCU polling API requires that the fake writers also use it in order to test concurrent calls to all of the API members, especially start_poll_synchronize_srcu(). This commit makes rcu_torture_fakewriter() use all available blocking grace-period-w

[PATCH tip/core/rcu 08/17] rcutorture: Use torture_hrtimeout_jiffies() to avoid busy-waits

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Because rcu_torture_writer() and rcu_torture_fakewriter() predate hrtimers, they do timer-wheel-decoupled timed waits by using the timer-wheel-based schedule_timeout_interruptible() functions in conjunction with a random udelay()-based wait. This latter unnecessarily bur

[PATCH tip/core/rcu 07/17] torture: Add fuzzed hrtimer-based sleep functions

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds torture_hrtimeout_ns(), torture_hrtimeout_us(), torture_hrtimeout_ms(), torture_hrtimeout_jiffies(), and torture_hrtimeout_s(), each of which uses hrtimers to block for a fuzzed time interval. These functions are intended to be used by the various tortur

[PATCH tip/core/rcu 02/17] scftorture: Add debug output for wrong-CPU warning

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds the desired CPU, the actual CPU, and nr_cpu_ids to the wrong-CPU warning in scftorture_invoker(), the better to help with debugging. Signed-off-by: Paul E. McKenney --- kernel/scftorture.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) dif

[PATCH tip/core/rcu 03/17] refscale: Allow summarization of verbose output

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The refscale test prints enough per-kthread console output to provoke RCU CPU stall warnings on large systems. This commit therefore allows this output to be summarized. For example, the refscale.verbose_batched=32 boot parameter would causes only every 32nd line of out

[PATCH tip/core/rcu 01/17] rcutorture: Add testing for RCU's global memory ordering

2021-01-06 Thread paulmck
From: "Paul E. McKenney" RCU guarantees that anything seen by a given reader will also be seen after any grace period that must wait on that reader. This is very likely to hold based on inspection, but the advantage of having rcutorture do the inspecting is that rcutorture doesn't mind inspectin

[PATCH tip/core/rcu 04/17] rcutorture: Require entire stutter period be post-boot

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Currently, the rcu_torture_writer() function checks that all required grace periods elapse during a stutter interval, which is a multi-second time period during which the test load is removed. However, this check is suppressed during early boot (that is, before init is s

[PATCH tip/core/rcu 05/17] rcutorture: Make synctype[] and nsynctype be static global

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Full testing of the new SRCU polling API requires that the fake writers also use it in order to test concurrent calls to all of the API members, especially start_poll_synchronize_srcu(). This commit prepares the ground for this by making the synctype[] and nsynctype vari

[PATCH tip/core/rcu 12/17] torture: Throttle VERBOSE_TOROUT_*() output

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds kernel boot parameters torture.verbose_sleep_frequency and torture.verbose_sleep_duration, which allow VERBOSE_TOROUT_*() output to be throttled with periodic sleeps on large systems. Signed-off-by: Paul E. McKenney --- Documentation/admin-guide/kernel

[PATCH tip/core/rcu 11/17] torture: Make refscale throttle high-rate printk()s

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds a short delay for verbose_batched-throttled printk()s to further decrease console flooding. Signed-off-by: Paul E. McKenney --- kernel/rcu/refscale.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/refscale.c b/kernel/

[PATCH tip/core/rcu 15/17] torture: Maintain torture-specific set of CPUs-online books

2021-01-06 Thread paulmck
_CPUS until shortly after torture_onoff_init() is invoked. Reported-by: Frederic Weisbecker [ paulmck: Apply feedback from kernel test robot. ] Signed-off-by: Paul E. McKenney --- include/linux/torture.h | 5 + kernel/rcu/rcutorture.c | 4 ++-- kernel/torture.c| 16

[PATCH tip/core/rcu 01/20] torture: Make --kcsan specify lockdep

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The --kcsan argument to kvm.sh adds CONFIG_KCSAN_VERBOSE=y in order to get more detail from the KCSAN reports. However, this Kconfig option requires lockdep to be enabled. This commit therefore causes --kcsan to also enable lockdep. Signed-off-by: Paul E. McKenney ---

[PATCH tip/core/rcu 02/20] torture: Make kvm.sh "--dryrun sched" summarize number of batches

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Knowing the number of batches that kvm.sh will split a run into allows estimation of the duration of a test, give or take the number of builds. This commit therefore adds a line of output to "--dryrun sched" that gives the number of batches that will be run. Signed-off-b

[PATCH tip/core/rcu 03/20] torture: Make kvm.sh "--dryrun sched" summarize number of builds

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Knowing the number of builds that kvm.sh will split a run into allows estimation of the duration of a test, give or take build duration. This commit therefore adds a line of output to "--dryrun sched" that gives the number of builds that will be run. This excludes "build

[PATCH tip/core/rcu 04/20] torture: Allow kvm.sh --datestamp to specify subdirectories

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Scripts like kvm-check-branches.sh group runs under a single directory in resdir in order to allow easier retrospective analysis. However, they do this by letting kvm.sh create a directory as usual and then moving it after the run. This can be very confusing when lookin

[PATCH tip/core/rcu 06/20] torture: Add config2csv.sh script to compare torture scenarios

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds a config2csv.sh script that converts the specified torture-test scenarios' Kconfig options and kernel-boot parameters to .csv format. This allows easier comparison of scenarios when one fails and another does not. Signed-off-by: Paul E. McKenney --- .

[PATCH tip/core/rcu 17/20] torture: Simplify exit-code plumbing for kvm-recheck.sh and kvm-find-errors.sh

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit simplifies exit-code plumbing. It makes kvm-recheck.sh return the value 1 for a build error and 2 for a runtime error. It also makes kvm-find-errors.sh avoid checking runtime files for --build-only runs. Signed-off-by: Paul E. McKenney --- tools/testing/s

[PATCH tip/core/rcu 05/20] torture: Prepare for splitting qemu execution from kvm-test-1-run.sh

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Distributed execution of rcutorture is eased if the qemu execution can be split from the building of the kernel, as this allows target systems to be used that are not set up to build kernels. It also avoids issues with toolchain version skew across the cluster, aside of

[PATCH tip/core/rcu 16/20] torture: s/STOP/STOP.1/ to avoid scenario collision

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit changes the "STOP" file that is used to cleanly halt a running rcutorture run to "STOP.1" because no scenario directory will ever end with ".1". If there really was a scenario named "STOP", its directories would instead be named "STOP", "STOP.2", "STOP.3", an

[PATCH tip/core/rcu 07/20] tools/rcutorture: Make identify_qemu_vcpus() independent of local language

2021-01-06 Thread paulmck
From: Frederic Weisbecker The rcutorture scripts' identify_qemu_vcpus() function expects `lscpu` to have a "CPU: " line, for example: CPU(s): 8 But different local language settings can give different results: Processeur(s) : 8 As a result, identify_qemu_vcpus(

[PATCH tip/core/rcu 08/20] torture: Make kvm.sh "Test Summary" date be end of test

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Currently, the "date" command producing the output on the kvm.sh "Test Summary" line is executed at the beginning of the test, which produces a date that is less than helpful to someone wanting to know the duration of the test. This commit therefore defers this command's

[PATCH tip/core/rcu 11/20] torture: Make kvm.sh return failure upon build failure

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The kvm.sh script uses kvm-find-errors.sh to evaluate whether or not a build failed. Unfortunately, kvm-find-errors.sh returns success if there are no failed runs (including when there are no runs at all) even if there are build failures. This commit therefore makes kvm

[PATCH tip/core/rcu 19/20] torture: Allow standalone kvm-recheck.sh run detect --trust-make

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Normally, kvm-recheck.sh is run from kvm.sh, which provides the TORTURE_TRUST_MAKE environment variable that, if a non-empty string, indicates that the --trust-make command-line parameter has been passed to kvm.sh. If there was no --trust-make, kvm-recheck.sh insists tha

[PATCH tip/core/rcu 14/20] torture: Stop hanging on panic

2021-01-06 Thread paulmck
From: "Paul E. McKenney" By default, the "panic" kernel parameter is zero, which causes the kernel to loop indefinitely after a panic(). The rcutorture scripting will eventually kill the corresponding qemu process, but only after waiting for the full run duration plus a few minutes. This works,

[PATCH tip/core/rcu 18/20] torture: Remove "Failed to add ttynull console" false positive

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Commit 757055ae8ded ("init/console: Use ttynull as a fallback when there is no console") results in the string "Warning: Failed to add ttynull console. No stdin, stdout, and stderr for the init process!" appearing on the console, which the rcutorture scripting interprets

[PATCH tip/core/rcu 15/20] torture: Add --dryrun batches to help schedule a distributed run

2021-01-06 Thread paulmck
From: "Paul E. McKenney" When all of the remote systems have the same number of CPUs, one approach is to use one "--buildonly" run and one "--dryrun sched" run, and then distributing the batches out one per remote system. However, the output of "--dryrun sched" is not made for parsing, so this c

[PATCH tip/core/rcu 20/20] torture: Do Kconfig analysis only once per scenario

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Currently, if a scenario is repeated as in "--configs '4*TREE01'", the Kconfig analysis is performed for each occurrance (four times in this example) and each analysis places the exact same data into the exact same files. This is not really an issue in this repetition-fo

[PATCH tip/core/rcu 10/20] torture: Print run duration at end of kvm.sh execution

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Yes, you can mentally subtract the timestamps, but this commit makes the computer do this work. Signed-off-by: Paul E. McKenney --- .../testing/selftests/rcutorture/bin/functions.sh | 33 ++ tools/testing/selftests/rcutorture/bin/kvm.sh | 6 +

[PATCH tip/core/rcu 12/20] torture: Make kvm.sh include --kconfig arguments in CPU calculation

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Currently, passing something like "--kconfig CONFIG_NR_CPUS=2" to kvm.sh has no effect on scenario scheduling. For scenarios that do not specify the number of CPUs, this can result in kvm.sh wastefully scheduling only one scenario at a time even when the --kconfig argume

[PATCH tip/core/rcu 13/20] torture: Add kvm.sh test summary to end of log file

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds the test summary to the end of the log in the top-level directory containing the kvm.sh test artifacts. While in the area, it adds the kvm.sh exit code to this test summary. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/kv

[PATCH tip/core/rcu 09/20] torture: Make kvm.sh arguments accumulate

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Given that kvm.sh in invoked from scripts, it is only natural for different levels of scripting to provide their own Kconfig option values, for example. Unfortunately, right now, the last such argument on the command line wins. This commit therefore makes the --bootargs

[PATCH tip/core/rcu 01/18] torture: Do Kconfig analysis only once per scenario

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Currently, if a scenario is repeated as in "--configs '4*TREE01'", the Kconfig analysis is performed for each occurrance (four times in this example) and each analysis places the exact same data into the exact same files. This is not really an issue in this repetition-fo

[PATCH tip/core/rcu 04/18] torture: Remove use of "eval" in torture.sh

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The bash "eval" command enables Bobby Tables attacks, which might not be a concern in torture testing by themselves, but one could imagine these combined with a cut-and-paste attack. This commit therefore gets rid of them. This comes at a price in terms of bash quoting

[PATCH tip/core/rcu 05/18] torture: Add "make allmodconfig" to torture.sh

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds the ability to do "make allmodconfig" to torture.sh, given that normal rcutorture runs do not normally catch missing exports. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 37 ++- 1 file cha

[PATCH tip/core/rcu 02/18] torture: Add torture.sh torture-everything script

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Although tailoring a specific set of kvm.sh runs has served rcutorture testing well over many years, it requires a relatively distraction-free environment, which is not always available. This commit therefore adds a prototype torture.sh script that by default tortures pr

[PATCH tip/core/rcu 06/18] torture: Auto-size SCF and scaling runs based on number of CPUs

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit improves torture.sh flexibility by autoscaling the number of CPUs to be used in variable-CPUs torture tests, including scftorture, refscale, rcuscale, and kvfree. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 19 +++

[PATCH tip/core/rcu 08/18] torture: Make torture.sh rcuscale and refscale deal with allmodconfig

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The .mod.c files created by allmodconfig builds interfers with the approach torture.sh uses to enumerate types of rcuscale and refscale runs. This commit therefore tightens the pattern matching to avoid this interference. Signed-off-by: Paul E. McKenney --- tools/test

[PATCH tip/core/rcu 07/18] torture: Enable torture.sh argument checking

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit uncomments the argument checking for the --duration argument to torture.sh. While in the area, it also corrects the duration units from seconds to minutes. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 10 +

[PATCH tip/core/rcu 03/18] torture: Make torture.sh use common time-duration bash functions

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit makes torture.sh use the new bash functions get_starttime() and get_starttime_duration() created for kvm.sh. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 20 +++- 1 file changed, 7 insertions(+), 13

[PATCH tip/core/rcu 14/18] torture: Drop log.long generation from torture.sh

2021-01-06 Thread paulmck
From: "Paul E. McKenney" Now that kvm.sh puts all the relevant details in the "log" file, there is no need for torture.sh to generate a separate "log.long" file. This commit therefore drops this from torture.sh. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/tortur

[PATCH tip/core/rcu 09/18] torture: Make torture.sh refscale runs use verbose_batched module parameter

2021-01-06 Thread paulmck
From: "Paul E. McKenney" On large systems, the refscale printk() rate can overrun the file system's ability to accept console log messages. This commit therefore uses the new verbose_batched module parameter to rate-limit some of the higher-rate printk() calls. Signed-off-by: Paul E. McKenney

[PATCH tip/core/rcu 15/18] torture: Allow scenarios to be specified to torture.sh

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds --configs-rcutorture, --configs-locktorture, and --configs-scftorture arguments to torture.sh, allowing the desired set of scenarios to be passed to each. The default for each has been changed from a large-system-appropriate set to just CFLIST for each.

[PATCH tip/core/rcu 17/18] torture: Add --kcsan-kmake-arg to torture.sh for KCSAN

2021-01-06 Thread paulmck
From: "Paul E. McKenney" In 2020, running KCSAN often requires careful choice of compiler. This commit therefore adds a --kcsan-kmake-arg parameter to torture.sh to allow specifying (for example) "CC=clang" to the kernel build process to correctly build a KCSAN-enabled kernel. Signed-off-by: Pau

[PATCH tip/core/rcu 12/18] torture: Make torture.sh throttle VERBOSE_TOROUT_*() for refscale

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit causes torture.sh to use the torture.verbose_sleep_frequency kernel boot parameter to throttle verbose refscale output on large systems. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 2 +- 1 file changed, 1 insertio

[PATCH tip/core/rcu 11/18] torture: Make torture.sh allmodconfig retain and label output

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit places "---" markers in the torture.sh script's allmodconfig output, and uses "<<" to avoid overwriting earlier output from this build test. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 9 ++--- 1 file changed,

[PATCH tip/core/rcu 10/18] torture: Create doyesno helper function for torture.sh

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit saves a few lines of code by creating a doyesno helper bash function for argument parsing. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 78 ++- 1 file changed, 19 insertions(+), 59 deletions(-)

[PATCH tip/core/rcu 13/18] torture: Make torture.sh refuse to do zero-length runs

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit causes torture.sh to check for zero-length runs and to take the cowardly option of refusing to run them, logging its cowardice for later inspection. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/torture.sh | 25 +

[PATCH tip/core/rcu 18/18] torture: Compress KASAN vmlinux files

2021-01-06 Thread paulmck
From: "Paul E. McKenney" The sizes of vmlinux files built with KASAN enabled can approach a full gigabyte, which can result in disk overflow sooner rather than later. Fortunately, the xz command compresses them by almost an order of magnitude. This commit therefore uses xz to compress vmlinux fi

[PATCH tip/core/rcu 16/18] torture: Add command and results directory to torture.sh log

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit adds the command and arguments to the torture.sh log file, and also outputs the results directory. This latter allows impatient users to quickly find the results that are being generated by the current run. Signed-off-by: Paul E. McKenney --- tools/testing

[PATCH tip/core/rcu 2/2] random32: Re-enable KCSAN instrumentation

2021-01-06 Thread paulmck
From: Marco Elver Re-enable KCSAN instrumentation, now that KCSAN no longer relies on code in lib/random32.c. Signed-off-by: Marco Elver Signed-off-by: Paul E. McKenney --- lib/Makefile | 3 --- 1 file changed, 3 deletions(-) diff --git a/lib/Makefile b/lib/Makefile index afeff05..dc09208 10

[PATCH tip/core/rcu 1/2] kcsan: Rewrite kcsan_prandom_u32_max() without prandom_u32_state()

2021-01-06 Thread paulmck
From: Marco Elver Rewrite kcsan_prandom_u32_max() to not depend on code that might be instrumented, removing any dependency on lib/random32.c. The rewrite implements a simple linear congruential generator, that is sufficient for our purposes (for udelay() and skip_watch counter randomness). The

[PATCH tip/core/rcu 1/3] tools/memory-model: Tie acquire loads to reads-from

2021-01-06 Thread paulmck
From: "Paul E. McKenney" This commit explicitly makes the connection between acquire loads and the reads-from relation. It also adds an entry for happens-before, and refers to the corresponding section of explanation.txt. Reported-by: Boqun Feng Signed-off-by: Paul E. McKenney --- tools/memo

[PATCH tip/core/rcu 2/3] tools/memory-model: Remove redundant initialization in litmus tests

2021-01-06 Thread paulmck
From: Akira Yokosawa This is a revert of commit 1947bfcf81a9 ("tools/memory-model: Add types to litmus tests") with conflict resolutions. klitmus7 [1] is aware of default types of "int" and "int*". It accepts litmus tests for herd7 without extra type info unless non-"int" variables are reference

[PATCH tip/core/rcu 3/3] tools/memory-model: Fix typo in klitmus7 compatibility table

2021-01-06 Thread paulmck
From: Akira Yokosawa klitmus7 of herdtools7 7.48 or earlier depends on ACCESS_ONCE(), which was removed in Linux v4.15. Fix the obvious typo in the table. Fixes: d075a78a5ab1 ("tools/memory-model/README: Expand dependency of klitmus7") Signed-off-by: Akira Yokosawa Signed-off-by: Paul E. McKenn

[PATCH v2 tip/core/rcu 2/6] srcu: Provide internal interface to start a Tiny SRCU grace period

2020-11-20 Thread paulmck
From: "Paul E. McKenney" There is a need for a polling interface for SRCU grace periods. This polling needs to initiate an SRCU grace period without having to queue (and manage) a callback. This commit therefore splits the Tiny SRCU call_srcu() function into callback-queuing and start-grace-peri

[PATCH v2 tip/core/rcu 1/6] srcu: Make Tiny SRCU use multi-bit grace-period counter

2020-11-20 Thread paulmck
an/ Reported-by: Kent Overstreet [ paulmck: Fix __srcu_read_lock() idx computation Neeraj per Upadhyay. ] Signed-off-by: Paul E. McKenney --- include/linux/srcutiny.h | 4 ++-- kernel/rcu/srcutiny.c| 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/include/linux/srcut

[PATCH v2 tip/core/rcu 5/6] srcu: Provide polling interfaces for Tree SRCU grace periods

2020-11-20 Thread paulmck
value from either get_state_synchronize_srcu() or start_poll_synchronize_srcu() must be passed in to a later call to poll_state_synchronize_srcu(). Link: https://lore.kernel.org/rcu/20201112201547.gf3365...@moria.home.lan/ Reported-by: Kent Overstreet [ paulmck: Add EXPORT_SYMBOL_GPL() per kernel

[PATCH v2 tip/core/rcu 4/6] srcu: Provide polling interfaces for Tiny SRCU grace periods

2020-11-20 Thread paulmck
value from either get_state_synchronize_srcu() or start_poll_synchronize_srcu() must be passed in to a later call to poll_state_synchronize_srcu(). Link: https://lore.kernel.org/rcu/20201112201547.gf3365...@moria.home.lan/ Reported-by: Kent Overstreet [ paulmck: Add EXPORT_SYMBOL_GPL() per kernel

[PATCH v2 tip/core/rcu 3/6] srcu: Provide internal interface to start a Tree SRCU grace period

2020-11-20 Thread paulmck
From: "Paul E. McKenney" There is a need for a polling interface for SRCU grace periods. This polling needs to initiate an SRCU grace period without having to queue (and manage) a callback. This commit therefore splits the Tree SRCU __call_srcu() function into callback-initialization and queuing

[PATCH v2 tip/core/rcu 6/6] srcu: Document polling interfaces for Tree SRCU grace periods

2020-11-20 Thread paulmck
From: "Paul E. McKenney" This commit adds requirements documentation for the get_state_synchronize_srcu(), start_poll_synchronize_srcu(), and poll_state_synchronize_srcu() functions. Link: https://lore.kernel.org/rcu/20201112201547.gf3365...@moria.home.lan/ Reported-by: Kent Overstreet Signed-o

[PATCH tip/core/rcu 01/32] tools/memory-model: Fix data race detection for unordered store and load

2019-10-02 Thread paulmck
From: Alan Stern Currently the Linux Kernel Memory Model gives an incorrect response for the following litmus test: C plain-WWC {} P0(int *x) { WRITE_ONCE(*x, 2); } P1(int *x, int *y) { int r1; int r2; int r3; r1 = READ_ONCE(*x); if (r1 == 2) {

[PATCH tip/core/rcu 17/32] tools/memory-model: Make runlitmus.sh generate .litmus.out for --hw

2019-10-02 Thread paulmck
From: "Paul E. McKenney" In the absence of "Result:" comments, the runlitmus.sh script relies on litmus.out files from prior LKMM runs. This can be a bit user-hostile, so this commit makes runlitmus.sh generate any needed .litmus.out files that don't already exist. Signed-off-by: Paul E. McKenn

[PATCH tip/core/rcu 24/32] tools/memory-model: Fix scripting --jobs argument

2019-10-02 Thread paulmck
From: "Paul E. McKenney" The parseargs.sh regular expression for the --jobs argument incorrectly requires that the number of jobs be at least 10, that is, have at least two digits. This commit therefore adjusts this regular expression to allow single-digit numbers of jobs to be specified. Signe

[PATCH tip/core/rcu 05/32] tools/memory-model: Make judgelitmus.sh note timeouts

2019-10-02 Thread paulmck
From: "Paul E. McKenney" Currently, judgelitmus.sh treats timeouts (as in the "--timeout" argument) as "!!! Verification error". This can be misleading because it is quite possible that running the test longer would have produced a verification. This commit therefore changes judgelitmus.sh to ch

[PATCH tip/core/rcu 11/32] tools/memory-model: Make judgelitmus.sh handle hardware verifications

2019-10-02 Thread paulmck
From: "Paul E. McKenney" This commit makes the judgelitmus.sh script check the --hw argument (AKA the LKMM_HW_MAP_FILE environment variable) and to adjust its judgment for a run where a C-language litmus test has been translated to assembly and the assembly version verified. In this case, the as

[PATCH tip/core/rcu 02/32] tools/memory-model/Documentation: Fix typos in explanation.txt

2019-10-02 Thread paulmck
From: Alan Stern This patch fixes a few minor typos and improves word usage in a few places in the Linux Kernel Memory Model's explanation.txt file. Signed-off-by: Alan Stern Reviewed-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney --- tools/memory-model/Documentation/explanation

[PATCH tip/core/rcu 28/32] tools/memory-model: Repair parseargs.sh header comment

2019-10-02 Thread paulmck
From: "Paul E. McKenney" Signed-off-by: Paul E. McKenney --- tools/memory-model/scripts/parseargs.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/memory-model/scripts/parseargs.sh b/tools/memory-model/scripts/parseargs.sh index 7aa5875..08ded59 100755 --- a/tools/m

[PATCH tip/core/rcu 26/32] tools/memory-model: Make history-check scripts use mselect7

2019-10-02 Thread paulmck
From: "Paul E. McKenney" The history-check scripts currently use grep to ignore non-C-language litmus tests, which is a bit fragile. This commit therefore enlists the aid of "mselect7 -arch C", given Luc Maraget's recent modifications that allow mselect7 to operate in filter mode. This change r

[PATCH tip/core/rcu 32/32] tools/memory-model: Use "-unroll 0" to keep --hw runs finite

2019-10-02 Thread paulmck
From: "Paul E. McKenney" Litmus tests involving atomic operations produce LL/SC loops on a number of architectures, and unrolling these loops can result in excessive verification times or even stack overflows. This commit therefore uses the "-unroll 0" herd7 argument to avoid unrolling, on the g

[PATCH tip/core/rcu 29/32] tools/memory-model: Add checktheselitmus.sh to run specified litmus tests

2019-10-02 Thread paulmck
From: "Paul E. McKenney" This commit adds a checktheselitmus.sh script that runs the litmus tests specified on the command line. This is useful for verifying fixes to specific litmus tests. Signed-off-by: Paul E. McKenney --- tools/memory-model/scripts/README | 8 + tools/me

<    1   2   3   4   5   6   7   8   >