On Mon, Mar 24, 2025 at 01:01:53PM -0400, Joel Fernandes wrote:
> The rcu_seq_done_exact() function checks if a grace period has completed by
> comparing sequence numbers. It includes a guard band to handle sequence number
> wraparound, which was previously expressed using the magic number calculation
> '3 * RCU_SEQ_STATE_MASK + 1'.
>
> This magic number is not immediately obvious in terms of what it represents.
>
> Instead, the reason we need this tiny guardband is because of the lag between
> the setting of rcu_state.gp_seq_polled and root rnp's gp_seq in rcu_gp_init().
>
> This guardband needs to be at least 2 GPs worth of counts, to avoid
> recognizing
> the newly started GP as completed immediately, due to the following sequence
> which arises due to the delay between update of rcu_state.gp_seq_polled and
> root rnp's gp_seq:
>
> rnp->gp_seq = rcu_state.gp_seq = 0
>
> CPU 0 CPU 1
> ----- -----
> // rcu_state.gp_seq = 1
> rcu_seq_start(&rcu_state.gp_seq)
> // snap = 8
> snap =
> rcu_seq_snap(&rcu_state.gp_seq)
> // Two full GP differences
>
> rcu_seq_done_exact(&rnp->gp_seq, snap)
> // rnp->gp_seq = 1
> WRITE_ONCE(rnp->gp_seq, rcu_state.gp_seq);
>
> This can happen due to get_state_synchronize_rcu_full() sampling
> rcu_state.gp_seq_polled, however the poll_state_synchronize_rcu_full()
> sampling the root rnp's gp_seq. The delay between the update of the 2
> counters occurs in rcu_gp_init() during which the counters briefly go
> out of sync.
>
> Make the guardband explictly 2 GPs. This improves code readability and
> maintainability by making the intent clearer as well.
>
> Suggested-by: Frederic Weisbecker <frede...@kernel.org>
> Signed-off-by: Joel Fernandes <joelagn...@nvidia.com>
One concern is that a small error anywhere in the code could cause this
minimal guard band to be too small. This is not a problem for some
use cases (rcu_barrier() just does an extra operation, and normal grace
periods are protected from forever-idle CPUs by ->gpwrap), but could be
an issue on 32-bit systems for user of polled RCU grace periods.
In contrast, making the guard band a bit longer than it needs to be
has little or no downside.
Thanx, Paul
> ---
> kernel/rcu/rcu.h | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
> index eed2951a4962..5e1ee570bb27 100644
> --- a/kernel/rcu/rcu.h
> +++ b/kernel/rcu/rcu.h
> @@ -57,6 +57,9 @@
> /* Low-order bit definition for polled grace-period APIs. */
> #define RCU_GET_STATE_COMPLETED 0x1
>
> +/* A complete grace period count */
> +#define RCU_SEQ_GP (RCU_SEQ_STATE_MASK + 1)
> +
> extern int sysctl_sched_rt_runtime;
>
> /*
> @@ -162,7 +165,7 @@ static inline bool rcu_seq_done_exact(unsigned long *sp,
> unsigned long s)
> {
> unsigned long cur_s = READ_ONCE(*sp);
>
> - return ULONG_CMP_GE(cur_s, s) || ULONG_CMP_LT(cur_s, s - (3 *
> RCU_SEQ_STATE_MASK + 1));
> + return ULONG_CMP_GE(cur_s, s) || ULONG_CMP_LT(cur_s, s - (2 *
> RCU_SEQ_GP));
> }
>
> /*
> --
> 2.43.0
>
