• Peter Zijlstra's avatar
    fs/locks: Replace lg_global with a percpu-rwsem · aba37660
    Peter Zijlstra authored
    Replace the global part of the lglock with a percpu-rwsem.
    Since fcl_lock is a spinlock and itself nests under i_lock, which too
    is a spinlock we cannot acquire sleeping locks at
    We can however wrap all fcl_lock acquisitions with percpu_down_read
    such that all invocations of locks_{insert,remove}_global_locks() have
    that read lock held.
    This allows us to replace the lg_global part of the lglock with the
    write side of the rwsem.
    In the absense of writers, percpu_{down,up}_read() are free of atomic
    instructions. This further avoids the very long preempt-disable
    regions caused by lglock on larger machines.
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Al Viro <viro@ZenIV.linux.org.uk>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: dave@stgolabs.net
    Cc: der.herr@hofr.at
    Cc: paulmck@linux.vnet.ibm.com
    Cc: riel@redhat.com
    Cc: tj@kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
locks.c 72.1 KB