rcu: Make idle entry report expedited quiescent states
authorPaul E. McKenney <paulmck@kernel.org>
Wed, 29 Sep 2021 18:09:34 +0000 (11:09 -0700)
committerPaul E. McKenney <paulmck@kernel.org>
Wed, 8 Dec 2021 00:22:22 +0000 (16:22 -0800)
commit790da248978a0722d92d1471630c881704f7eb0d
tree1c78e09b356d5dec37b650046fe141c03418715c
parent147f04b14adde831eb4a0a1e378667429732f9e8
rcu: Make idle entry report expedited quiescent states

In non-preemptible kernels, an unfortunately timed expedited grace period
can result in the rcu_exp_handler() IPI handler setting the rcu_data
structure's cpu_no_qs.b.exp field just as the target CPU enters idle.
There are situations in which this field will not be checked until after
that CPU exits idle.  The resulting grace-period latency does not qualify
as "expedited".

This commit therefore checks this field upon non-preemptible idle entry in
the rcu_preempt_deferred_qs() function.  It also qualifies the rcu_core()
preempt_count() check with IS_ENABLED(CONFIG_PREEMPT_COUNT) to prevent
false-positive quiescent states from count-free kernels.

Reported-by: Neeraj Upadhyay <neeraju@codeaurora.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
kernel/rcu/tree.c
kernel/rcu/tree_plugin.h