Skip to content
  • Boqun Feng's avatar
    rcu: Don't disable preemption for Tiny and Tree RCU readers · bb73c52b
    Boqun Feng authored
    
    
    Because preempt_disable() maps to barrier() for non-debug builds,
    it forces the compiler to spill and reload registers.  Because Tree
    RCU and Tiny RCU now only appear in CONFIG_PREEMPT=n builds, these
    barrier() instances generate needless extra code for each instance of
    rcu_read_lock() and rcu_read_unlock().  This extra code slows down Tree
    RCU and bloats Tiny RCU.
    
    This commit therefore removes the preempt_disable() and preempt_enable()
    from the non-preemptible implementations of __rcu_read_lock() and
    __rcu_read_unlock(), respectively.  However, for debug purposes,
    preempt_disable() and preempt_enable() are still invoked if
    CONFIG_PREEMPT_COUNT=y, because this allows detection of sleeping inside
    atomic sections in non-preemptible kernels.
    
    However, Tiny and Tree RCU operates by coalescing all RCU read-side
    critical sections on a given CPU that lie between successive quiescent
    states.  It is therefore necessary to compensate for removing barriers
    from __rcu_read_lock() and __rcu_read_unlock() by adding them to a
    couple of the RCU functions invoked during quiescent states, namely to
    rcu_all_qs() and rcu_note_context_switch().  However, note that the latter
    is more paranoia than necessity, at least until link-time optimizations
    become more aggressive.
    
    This is based on an earlier patch by Paul E. McKenney, fixing
    a bug encountered in kernels built with CONFIG_PREEMPT=n and
    CONFIG_PREEMPT_COUNT=y.
    
    Signed-off-by: default avatarBoqun Feng <boqun.feng@gmail.com>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    bb73c52b