1. 07 Nov, 2015 1 commit
    • Dmitry Vyukov's avatar
      lib/llist.c: fix data race in llist_del_first · 2cf12f82
      Dmitry Vyukov authored
      llist_del_first reads entry->next, but it did not acquire visibility over
      the entry node.  As the result it can get a stale value of entry->next
      (e.g.  NULL or whatever garbage was there before the appending thread
      wrote correct value).  And then commit that value as llist head with
      cmpxchg.  That will corrupt llist.
      Note there is a control-dependency between read of head->first and read of
      entry->next, but it does not make the code correct.  Kernel memory model
      unambiguously says: "A load-load control dependency requires a full read
      memory barrier".
      Use smp_load_acquire to acquire visibility over the entry node.
      The data race was found with KernelThreadSanitizer (KTSAN).
      Here is an example of KTSAN report:
      ThreadSanitizer: data-race in llist_del_first
      Read of size 1 by thread T389 (K2630, CPU0):
       [<ffffffff8156b8a9>] llist_del_first+0x39/0x70 lib/llist.c:74
       [<     inlined    >] tty_buffer_alloc drivers/tty/tty_buffer.c:181
       [<ffffffff81664af4>] __tty_buffer_request_room+0xb4/0x250 drivers/tty/tty_buffer.c:292
       [<ffffffff81664e6c>] tty_insert_flip_string_fixed_flag+0x6c/0x150 drivers/tty/tty_buffer.c:337
       [<     inlined    >] tty_insert_flip_string include/linux/tty_flip.h:35
       [<ffffffff81667422>] pty_write+0x72/0xc0 drivers/tty/pty.c:110
       [<     inlined    >] process_output_block drivers/tty/n_tty.c:611
       [<ffffffff8165c016>] n_tty_write+0x346/0x7f0 drivers/tty/n_tty.c:2401
       [<     inlined    >] do_tty_write drivers/tty/tty_io.c:1159
       [<ffffffff816568df>] tty_write+0x21f/0x3f0 drivers/tty/tty_io.c:1245
       [<ffffffff8125f00f>] __vfs_write+0x5f/0x1f0 fs/read_write.c:489
       [<ffffffff8125ff8f>] vfs_write+0xef/0x280 fs/read_write.c:538
       [<     inlined    >] SYSC_write fs/read_write.c:585
       [<ffffffff81261390>] SyS_write+0x70/0xe0 fs/read_write.c:577
       [<ffffffff81ee862e>] entry_SYSCALL_64_fastpath+0x12/0x71 arch/x86/entry/entry_64.S:186
      Previous write of size 8 by thread T226 (K761, CPU0):
       [<ffffffff8156b832>] llist_add_batch+0x32/0x70 lib/llist.c:44 (discriminator 16)
       [<     inlined    >] llist_add include/linux/llist.h:180
       [<ffffffff816649fc>] tty_buffer_free+0x6c/0xb0 drivers/tty/tty_buffer.c:221
       [<ffffffff816651e7>] flush_to_ldisc+0x107/0x300 drivers/tty/tty_buffer.c:514
       [<ffffffff810b20ee>] process_one_work+0x47e/0x930 kernel/workqueue.c:2036
       [<ffffffff810b2650>] worker_thread+0xb0/0x900 kernel/workqueue.c:2170
       [<ffffffff810bbe20>] kthread+0x150/0x170 kernel/kthread.c:209
       [<ffffffff81ee8a1f>] ret_from_fork+0x3f/0x70 arch/x86/entry/entry_64.S:526
      Signed-off-by: default avatarDmitry Vyukov <dvyukov@google.com>
      Reviewed-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
      Cc: Huang Ying <ying.huang@intel.com>
      Cc: Konstantin Serebryany <kcc@google.com>
      Cc: Andrey Konovalov <andreyknvl@google.com>
      Cc: Alexander Potapenko <glider@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
  2. 13 Feb, 2015 1 commit
  3. 15 Nov, 2013 2 commits
  4. 13 Jul, 2013 1 commit
    • Oleg Nesterov's avatar
      llist: fix/simplify llist_add() and llist_add_batch() · fb4214db
      Oleg Nesterov authored
      1. This is mostly theoretical, but llist_add*() need ACCESS_ONCE().
         Otherwise it is not guaranteed that the first cmpxchg() uses the
         same value for old_entry and new_last->next.
      2. These helpers cache the result of cmpxchg() and read the initial
         value of head->first before the main loop. I do not think this
         makes sense. In the likely case cmpxchg() succeeds, otherwise
         it doesn't hurt to reload head->first.
         I think it would be better to simplify the code and simply read
         ->first before cmpxchg().
      Signed-off-by: default avatarOleg Nesterov <oleg@redhat.com>
      Cc: Al Viro <viro@zeniv.linux.org.uk>
      Cc: Andrey Vagin <avagin@openvz.org>
      Cc: "Eric W. Biederman" <ebiederm@xmission.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Huang Ying <ying.huang@intel.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
  5. 28 Mar, 2012 1 commit
  6. 07 Mar, 2012 1 commit
  7. 04 Oct, 2011 5 commits
  8. 03 Aug, 2011 1 commit
    • Huang Ying's avatar
      lib, Add lock-less NULL terminated single list · f49f23ab
      Huang Ying authored
      Cmpxchg is used to implement adding new entry to the list, deleting
      all entries from the list, deleting first entry of the list and some
      other operations.
      Because this is a single list, so the tail can not be accessed in O(1).
      If there are multiple producers and multiple consumers, llist_add can
      be used in producers and llist_del_all can be used in consumers.  They
      can work simultaneously without lock.  But llist_del_first can not be
      used here.  Because llist_del_first depends on list->first->next does
      not changed if list->first is not changed during its operation, but
      llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
      llist_add) sequence in another consumer may violate that.
      If there are multiple producers and one consumer, llist_add can be
      used in producers and llist_del_all or llist_del_first can be used in
      the consumer.
      This can be summarized as follow:
                 |   add    | del_first |  del_all
       add       |    -     |     -     |     -
       del_first |          |     L     |     L
       del_all   |          |           |     -
      Where "-" stands for no lock is needed, while "L" stands for lock is
      The list entries deleted via llist_del_all can be traversed with
      traversing function such as llist_for_each etc.  But the list entries
      can not be traversed safely before deleted from the list.  The order
      of deleted entries is from the newest to the oldest added one.  If you
      want to traverse from the oldest to the newest, you must reverse the
      order by yourself before traversing.
      The basic atomic operation of this list is cmpxchg on long.  On
      architectures that don't have NMI-safe cmpxchg implementation, the
      list can NOT be used in NMI handler.  So code uses the list in NMI
      handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
      Signed-off-by: default avatarHuang Ying <ying.huang@intel.com>
      Reviewed-by: default avatarAndi Kleen <ak@linux.intel.com>
      Reviewed-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLen Brown <len.brown@intel.com>