rcu: Simplify rcu_read_unlock_special() deferred wakeups

In !use_softirq runs, we clearly cannot rely on raise_softirq() and
its lightweight bit setting, so we must instead do some form of wakeup.
In the absence of a self-IPI when interrupts are disabled, these wakeups
can be delayed until the next interrupt occurs.  This means that calling
invoke_rcu_core() doesn't actually do any expediting.

In this case, it is better to take the "else" clause, which sets the
current CPU's resched bits and, if there is an expedited grace period
in flight, uses IRQ-work to force the needed self-IPI.  This commit
therefore removes the "else if" clause that calls invoke_rcu_core().

Reported-by: Scott Wood <swood@redhat.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index acb225023ed1939a23ac143f0d713405764075ac..3f0701e860e41048864e4a33b7b3645d5f636e3c 100644 (file)
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -631,17 +631,12 @@ static void rcu_read_unlock_special(struct task_struct *t)
                        // Using softirq, safe to awaken, and we get
                        // no help from enabling irqs, unlike bh/preempt.
                        raise_softirq_irqoff(RCU_SOFTIRQ);
-               } else if (exp && irqs_were_disabled && !use_softirq &&
-                          !t->rcu_read_unlock_special.b.deferred_qs) {
-                       // Safe to awaken and we get no help from enabling
-                       // irqs, unlike bh/preempt.
-                       invoke_rcu_core();
                } else {
                        // Enabling BH or preempt does reschedule, so...
                        // Also if no expediting or NO_HZ_FULL, slow is OK.
                        set_tsk_need_resched(current);
                        set_preempt_need_resched();
-                       if (IS_ENABLED(CONFIG_IRQ_WORK) &&
+                       if (IS_ENABLED(CONFIG_IRQ_WORK) && irqs_were_disabled &&
                            !rdp->defer_qs_iw_pending && exp) {
                                // Get scheduler to re-evaluate and call hooks.
                                // If !IRQ_WORK, FQS scan will eventually IPI.