• Thomas Gleixner's avatar
    x86/speculation: Prepare for conditional IBPB in switch_mm() · cbca99b9
    Thomas Gleixner authored
    commit 4c71a2b6fd7e42814aa68a6dec88abf3b42ea573 upstream
    
    The IBPB speculation barrier is issued from switch_mm() when the kernel
    switches to a user space task with a different mm than the user space task
    which ran last on the same CPU.
    
    An additional optimization is to avoid IBPB when the incoming task can be
    ptraced by the outgoing task. This optimization only works when switching
    directly between two user space tasks. When switching from a kernel task to
    a user space task the optimization fails because the previous task cannot
    be accessed anymore. So for quite some scenarios the optimization is just
    adding overhead.
    
    The upcoming conditional IBPB support will issue IBPB only for user space
    tasks which have the TIF_SPEC_IB bit set. This requires to handle the
    following cases:
    
      1) Switch from a user space task (potential attacker) which has
         TIF_SPEC_IB set to a user space task (potential victim) which has
         TIF_SPEC_IB not set.
    
      2) Switch from a user space task (potential attacker) which has
         TIF_SPEC_IB not set to a user space task (potential victim) which has
         TIF_SPEC_IB set.
    
    This needs to be optimized for the case where the IBPB can be avoided when
    only kernel threads ran in between user space tasks which belong to the
    same process.
    
    The current check whether two tasks belong to the same context is using the
    tasks context id. While correct, it's simpler to use the mm pointer because
    it allows to mangle the TIF_SPEC_IB bit into it. The context id based
    mechanism requires extra storage, which creates worse code.
    
    When a task is scheduled out its TIF_SPEC_IB bit is mangled as bit 0 into
    the per CPU storage which is used to track the last user space mm which was
    running on a CPU. This bit can be used together with the TIF_SPEC_IB bit of
    the incoming task to make the decision whether IBPB needs to be issued or
    not to cover the two cases above.
    
    As conditional IBPB is going to be the default, remove the dubious ptrace
    check for the IBPB always case and simply issue IBPB always when the
    process changes.
    
    Move the storage to a different place in the struct as the original one
    created a hole.
    Signed-off-by: 's avatarThomas Gleixner <tglx@linutronix.de>
    Reviewed-by: 's avatarIngo Molnar <mingo@kernel.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Andy Lutomirski <luto@kernel.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Jiri Kosina <jkosina@suse.cz>
    Cc: Tom Lendacky <thomas.lendacky@amd.com>
    Cc: Josh Poimboeuf <jpoimboe@redhat.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: David Woodhouse <dwmw@amazon.co.uk>
    Cc: Tim Chen <tim.c.chen@linux.intel.com>
    Cc: Andi Kleen <ak@linux.intel.com>
    Cc: Dave Hansen <dave.hansen@intel.com>
    Cc: Casey Schaufler <casey.schaufler@intel.com>
    Cc: Asit Mallick <asit.k.mallick@intel.com>
    Cc: Arjan van de Ven <arjan@linux.intel.com>
    Cc: Jon Masters <jcm@redhat.com>
    Cc: Waiman Long <longman9394@gmail.com>
    Cc: Greg KH <gregkh@linuxfoundation.org>
    Cc: Dave Stewart <david.c.stewart@intel.com>
    Cc: Kees Cook <keescook@chromium.org>
    Cc: stable@vger.kernel.org
    Link: https://lkml.kernel.org/r/20181125185005.466447057@linutronix.deSigned-off-by: 's avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
    cbca99b9
tlbflush.h 17 KB