1. 20 May, 2016 1 commit
  2. 28 May, 2015 1 commit
    • Dave Chinner's avatar
      percpu_counter: batch size aware __percpu_counter_compare() · 80188b0d
      Dave Chinner authored
      XFS uses non-stanard batch sizes for avoiding frequent global
      counter updates on it's allocated inode counters, as they increment
      or decrement in batches of 64 inodes. Hence the standard percpu
      counter batch of 32 means that the counter is effectively a global
      counter. Currently Xfs uses a batch size of 128 so that it doesn't
      take the global lock on every single modification.
      
      However, Xfs also needs to compare accurately against zero, which
      means we need to use percpu_counter_compare(), and that has a
      hard-coded batch size of 32, and hence will spuriously fail to
      detect when it is supposed to use precise comparisons and hence
      the accounting goes wrong.
      
      Add __percpu_counter_compare() to take a custom batch size so we can
      use it sanely in XFS and factor percpu_counter_compare() to use it.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Acked-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarDave Chinner <david@fromorbit.com>
      80188b0d
  3. 08 Sep, 2014 2 commits
    • Tejun Heo's avatar
      percpu_counter: add @gfp to percpu_counter_init() · 908c7f19
      Tejun Heo authored
      Percpu allocator now supports allocation mask.  Add @gfp to
      percpu_counter_init() so that !GFP_KERNEL allocation masks can be used
      with percpu_counters too.
      
      We could have left percpu_counter_init() alone and added
      percpu_counter_init_gfp(); however, the number of users isn't that
      high and introducing _gfp variants to all percpu data structures would
      be quite ugly, so let's just do the conversion.  This is the one with
      the most users.  Other percpu data structures are a lot easier to
      convert.
      
      This patch doesn't make any functional difference.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarJan Kara <jack@suse.cz>
      Acked-by: default avatar"David S. Miller" <davem@davemloft.net>
      Cc: x86@kernel.org
      Cc: Jens Axboe <axboe@kernel.dk>
      Cc: "Theodore Ts'o" <tytso@mit.edu>
      Cc: Alexander Viro <viro@zeniv.linux.org.uk>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      908c7f19
    • Tejun Heo's avatar
      percpu_counter: make percpu_counters_lock irq-safe · ebd8fef3
      Tejun Heo authored
      percpu_counter is scheduled to grow @gfp support to allow atomic
      initialization.  This patch makes percpu_counters_lock irq-safe so
      that it can be safely used from atomic contexts.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      ebd8fef3
  4. 08 Apr, 2014 1 commit
    • Jens Axboe's avatar
      lib/percpu_counter.c: fix bad percpu counter state during suspend · e39435ce
      Jens Axboe authored
      I got a bug report yesterday from Laszlo Ersek in which he states that
      his kvm instance fails to suspend.  Laszlo bisected it down to this
      commit 1cf7e9c6 ("virtio_blk: blk-mq support") where virtio-blk is
      converted to use the blk-mq infrastructure.
      
      After digging a bit, it became clear that the issue was with the queue
      drain.  blk-mq tracks queue usage in a percpu counter, which is
      incremented on request alloc and decremented when the request is freed.
      The initial hunt was for an inconsistency in blk-mq, but everything
      seemed fine.  In fact, the counter only returned crazy values when
      suspend was in progress.
      
      When a CPU is unplugged, the percpu counters merges that CPU state with
      the general state.  blk-mq takes care to register a hotcpu notifier with
      the appropriate priority, so we know it runs after the percpu counter
      notifier.  However, the percpu counter notifier only merges the state
      when the CPU is fully gone.  This leaves a state transition where the
      CPU going away is no longer in the online mask, yet it still holds
      private values.  This means that in this state, percpu_counter_sum()
      returns invalid results, and the suspend then hangs waiting for
      abs(dead-cpu-value) requests to complete which of course will never
      happen.
      
      Fix this by clearing the state earlier, so we never have a case where
      the CPU isn't in online mask but still holds private state.  This bug
      has been there since forever, I guess we don't have a lot of users where
      percpu counters needs to be reliable during the suspend cycle.
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      Reported-by: default avatarLaszlo Ersek <lersek@redhat.com>
      Tested-by: default avatarLaszlo Ersek <lersek@redhat.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      e39435ce
  5. 07 Apr, 2014 1 commit
    • Jens Axboe's avatar
      percpu_counter: fix bad counter state during suspend · 60b0ea12
      Jens Axboe authored
      I got a bug report yesterday from Laszlo Ersek <lersek@xxxxxxxxxx>, in
      which he states that his kvm instance fails to suspend. He Laszlo
      bisected it down to this commit:
      
      commit 1cf7e9c6
      Author: Jens Axboe <axboe@xxxxxxxxx>
      Date: Fri Nov 1 10:52:52 2013 -0600
      
      virtio_blk: blk-mq support
      
      where virtio-blk is converted to use the blk-mq infrastructure. After
      digging a bit, it became clear that the issue was with the queue drain.
      blk-mq tracks queue usage in a percpu counter, which is incremented on
      request alloc and decremented when the request is freed. The initial
      hunt was for an inconsistency in blk-mq, but everything seemed fine. In
      fact, the counter only returned crazy values when suspend was in
      progress. When a CPU is unplugged, the percpu counters merges that CPU
      state with the general state. blk-mq takes care to register a hotcpu
      notifier with the appropriate priority, so we know it runs after the
      percpu counter notifier. However, the percpu counter notifier only
      merges the state when the CPU is fully gone. This leaves a state
      transition where the CPU going away is no longer in the online mask, yet
      it still holds private values. This means that in this state,
      percpu_counter_sum() returns invalid results, and the suspend then hangs
      waiting for abs(dead-cpu-value) requests to complete which of course
      will never happen.
      
      Fix this by clearing the state earlier, so we never have a case where
      the CPU isn't in online mask but still holds private state. This bug has
      been there since forever, I guess we don't have a lot of users where
      percpu counters needs to be reliable during the suspend cycle.
      
      Reported-by: <lersek@redhat.com>
      Tested-by: default avatarLaszlo Ersek <lersek@redhat.com>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      60b0ea12
  6. 17 Jan, 2014 1 commit
  7. 15 Jan, 2014 1 commit
    • Ming Lei's avatar
      lib/percpu_counter.c: fix __percpu_counter_add() · 74e72f89
      Ming Lei authored
      __percpu_counter_add() may be called in softirq/hardirq handler (such
      as, blk_mq_queue_exit() is typically called in hardirq/softirq handler),
      so we need to call this_cpu_add()(irq safe helper) to update percpu
      counter, otherwise counts may be lost.
      
      This fixes the problem that 'rmmod null_blk' hangs in blk_cleanup_queue()
      because of miscounting of request_queue->mq_usage_counter.
      
      This patch is the v1 of previous one of "lib/percpu_counter.c:
      disable local irq when updating percpu couter", and takes Andrew's
      approach which may be more efficient for ARCHs(x86, s390) that
      have optimized this_cpu_add().
      Signed-off-by: default avatarMing Lei <tom.leiming@gmail.com>
      Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
      Cc: Shaohua Li <shli@fusionio.com>
      Cc: Jens Axboe <axboe@kernel.dk>
      Cc: Fan Du <fan.du@windriver.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      74e72f89
  8. 25 Oct, 2013 1 commit
    • Shaohua Li's avatar
      percpu_counter: make APIs irq safe · 098faf58
      Shaohua Li authored
      In my usage, sometimes the percpu APIs are called with irq locked,
      sometimes not. lockdep complains there is potential deadlock. Let's
      always use percpucounter lock in irq safe way. There should be no
      performance penality, as all those are slow code path.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarShaohua Li <shli@fusionio.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      098faf58
  9. 14 Jul, 2013 1 commit
    • Paul Gortmaker's avatar
      kernel: delete __cpuinit usage from all core kernel files · 0db0628d
      Paul Gortmaker authored
      The __cpuinit type of throwaway sections might have made sense
      some time ago when RAM was more constrained, but now the savings
      do not offset the cost and complications.  For example, the fix in
      commit 5e427ec2 ("x86: Fix bit corruption at CPU resume time")
      is a good example of the nasty type of bugs that can be created
      with improper use of the various __init prefixes.
      
      After a discussion on LKML[1] it was decided that cpuinit should go
      the way of devinit and be phased out.  Once all the users are gone,
      we can then finally remove the macros themselves from linux/init.h.
      
      This removes all the uses of the __cpuinit macros from C files in
      the core kernel directories (kernel, init, lib, mm, and include)
      that don't really have a specific maintainer.
      
      [1] https://lkml.org/lkml/2013/5/20/589Signed-off-by: default avatarPaul Gortmaker <paul.gortmaker@windriver.com>
      0db0628d
  10. 03 Jul, 2013 1 commit
  11. 31 Jul, 2012 1 commit
  12. 01 Nov, 2011 1 commit
  13. 13 Sep, 2011 1 commit
  14. 17 Dec, 2010 1 commit
    • Christoph Lameter's avatar
      percpucounter: Optimize __percpu_counter_add a bit through the use of this_cpu() options. · 819a72af
      Christoph Lameter authored
      The this_cpu_* options can be used to optimize __percpu_counter_add a bit. Avoids
      some address arithmetic and saves 12 bytes.
      
      Before:
      
      
      00000000000001d3 <__percpu_counter_add>:
       1d3:	55                   	push   %rbp
       1d4:	48 89 e5             	mov    %rsp,%rbp
       1d7:	41 55                	push   %r13
       1d9:	41 54                	push   %r12
       1db:	53                   	push   %rbx
       1dc:	48 89 fb             	mov    %rdi,%rbx
       1df:	48 83 ec 08          	sub    $0x8,%rsp
       1e3:	4c 8b 67 30          	mov    0x30(%rdi),%r12
       1e7:	65 4c 03 24 25 00 00 	add    %gs:0x0,%r12
       1ee:	00 00
       1f0:	4d 63 2c 24          	movslq (%r12),%r13
       1f4:	48 63 c2             	movslq %edx,%rax
       1f7:	49 01 f5             	add    %rsi,%r13
       1fa:	49 39 c5             	cmp    %rax,%r13
       1fd:	7d 0a                	jge    209 <__percpu_counter_add+0x36>
       1ff:	f7 da                	neg    %edx
       201:	48 63 d2             	movslq %edx,%rdx
       204:	49 39 d5             	cmp    %rdx,%r13
       207:	7f 1e                	jg     227 <__percpu_counter_add+0x54>
       209:	48 89 df             	mov    %rbx,%rdi
       20c:	e8 00 00 00 00       	callq  211 <__percpu_counter_add+0x3e>
       211:	4c 01 6b 18          	add    %r13,0x18(%rbx)
       215:	48 89 df             	mov    %rbx,%rdi
       218:	41 c7 04 24 00 00 00 	movl   $0x0,(%r12)
       21f:	00
       220:	e8 00 00 00 00       	callq  225 <__percpu_counter_add+0x52>
       225:	eb 04                	jmp    22b <__percpu_counter_add+0x58>
       227:	45 89 2c 24          	mov    %r13d,(%r12)
       22b:	5b                   	pop    %rbx
       22c:	5b                   	pop    %rbx
       22d:	41 5c                	pop    %r12
       22f:	41 5d                	pop    %r13
       231:	c9                   	leaveq
       232:	c3                   	retq
      
      
      After:
      
      00000000000001d3 <__percpu_counter_add>:
       1d3:	55                   	push   %rbp
       1d4:	48 63 ca             	movslq %edx,%rcx
       1d7:	48 89 e5             	mov    %rsp,%rbp
       1da:	41 54                	push   %r12
       1dc:	53                   	push   %rbx
       1dd:	48 89 fb             	mov    %rdi,%rbx
       1e0:	48 8b 47 30          	mov    0x30(%rdi),%rax
       1e4:	65 44 8b 20          	mov    %gs:(%rax),%r12d
       1e8:	4d 63 e4             	movslq %r12d,%r12
       1eb:	49 01 f4             	add    %rsi,%r12
       1ee:	49 39 cc             	cmp    %rcx,%r12
       1f1:	7d 0a                	jge    1fd <__percpu_counter_add+0x2a>
       1f3:	f7 da                	neg    %edx
       1f5:	48 63 d2             	movslq %edx,%rdx
       1f8:	49 39 d4             	cmp    %rdx,%r12
       1fb:	7f 21                	jg     21e <__percpu_counter_add+0x4b>
       1fd:	48 89 df             	mov    %rbx,%rdi
       200:	e8 00 00 00 00       	callq  205 <__percpu_counter_add+0x32>
       205:	4c 01 63 18          	add    %r12,0x18(%rbx)
       209:	48 8b 43 30          	mov    0x30(%rbx),%rax
       20d:	48 89 df             	mov    %rbx,%rdi
       210:	65 c7 00 00 00 00 00 	movl   $0x0,%gs:(%rax)
       217:	e8 00 00 00 00       	callq  21c <__percpu_counter_add+0x49>
       21c:	eb 04                	jmp    222 <__percpu_counter_add+0x4f>
       21e:	65 44 89 20          	mov    %r12d,%gs:(%rax)
       222:	5b                   	pop    %rbx
       223:	41 5c                	pop    %r12
       225:	c9                   	leaveq
       226:	c3                   	retq
      Reviewed-by: default avatarPekka Enberg <penberg@kernel.org>
      Reviewed-by: default avatarTejun Heo <tj@kernel.org>
      Reviewed-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Acked-by: default avatarH. Peter Anvin <hpa@zytor.com>
      Signed-off-by: default avatarChristoph Lameter <cl@linux.com>
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      819a72af
  15. 26 Oct, 2010 3 commits
    • Christoph Lameter's avatar
      percpu_counter: use this_cpu_ptr() instead of per_cpu_ptr() · ea00c30b
      Christoph Lameter authored
      this_cpu_ptr() avoids an array lookup and can use the percpu offset of the
      local cpu directly.
      Signed-off-by: default avatarChristoph Lameter <cl@linux.com>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Tejun Heo <tj@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      ea00c30b
    • Tejun Heo's avatar
      percpu_counter: add debugobj support · e2852ae8
      Tejun Heo authored
      All percpu counters are linked to a global list on initialization and
      removed from it on destruction.  The list is walked during CPU up/down.
      If a percpu counter is freed without being properly destroyed, the system
      will oops only on the next CPU up/down making it pretty nasty to track
      down.  This patch adds debugobj support for percpu counters so that such
      problems can be found easily.
      
      As percpu counters don't make sense on stack and can't be statically
      initialized, debugobj support is pretty simple.  It's initialized and
      activated on counter initialization, and deactivatd and destroyed on
      counter destruction.  With this patch applied, the bug fixed by commit
      602586a8 (shmem: put_super must
      percpu_counter_destroy) triggers the following warning on tmpfs unmount
      and the system won't oops on the next cpu up/down operation.
      
       ------------[ cut here ]------------
       WARNING: at lib/debugobjects.c:259 debug_print_object+0x5c/0x70()
       Hardware name: Bochs
       ODEBUG: free active (active state 0) object type: percpu_counter
       Modules linked in:
       Pid: 3999, comm: umount Not tainted 2.6.36-rc2-work+ #5
       Call Trace:
        [<ffffffff81083f7f>] warn_slowpath_common+0x7f/0xc0
        [<ffffffff81084076>] warn_slowpath_fmt+0x46/0x50
        [<ffffffff813b45cc>] debug_print_object+0x5c/0x70
        [<ffffffff813b50e5>] debug_check_no_obj_freed+0x125/0x210
        [<ffffffff811577d3>] kfree+0xb3/0x2f0
        [<ffffffff81132edd>] shmem_put_super+0x1d/0x30
        [<ffffffff81162e96>] generic_shutdown_super+0x56/0xe0
        [<ffffffff81162f86>] kill_anon_super+0x16/0x60
        [<ffffffff81162ff7>] kill_litter_super+0x27/0x30
        [<ffffffff81163295>] deactivate_locked_super+0x45/0x60
        [<ffffffff81163cfa>] deactivate_super+0x4a/0x70
        [<ffffffff8117d446>] mntput_no_expire+0x86/0xe0
        [<ffffffff8117df7f>] sys_umount+0x6f/0x360
        [<ffffffff8103f01b>] system_call_fastpath+0x16/0x1b
       ---[ end trace cce2a341ba3611a7 ]---
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: Thomas Gleixner <tglxlinutronix.de>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      e2852ae8
    • Masanori ITOH's avatar
      percpu: fix list_head init bug in __percpu_counter_init() · 8474b591
      Masanori ITOH authored
      WARNING: at lib/list_debug.c:26 __list_add+0x3f/0x81()
      Hardware name: Express5800/B120a [N8400-085]
      list_add corruption. next->prev should be prev (ffffffff81a7ea00), but was dead000000200200. (next=ffff88080b872d58).
      Modules linked in: aoe ipt_MASQUERADE iptable_nat nf_nat autofs4 sunrpc bridge 8021q garp stp llc ipv6 cpufreq_ondemand acpi_cpufreq freq_table dm_round_robin dm_multipath kvm_intel kvm uinput lpfc scsi_transport_fc igb ioatdma scsi_tgt i2c_i801 i2c_core dca iTCO_wdt iTCO_vendor_support pcspkr shpchp megaraid_sas [last unloaded: aoe]
      Pid: 54, comm: events/3 Tainted: G        W  2.6.34-vanilla1 #1
      Call Trace:
      [<ffffffff8104bd77>] warn_slowpath_common+0x7c/0x94
      [<ffffffff8104bde6>] warn_slowpath_fmt+0x41/0x43
      [<ffffffff8120fd2e>] __list_add+0x3f/0x81
      [<ffffffff81212a12>] __percpu_counter_init+0x59/0x6b
      [<ffffffff810d8499>] bdi_init+0x118/0x17e
      [<ffffffff811f2c50>] blk_alloc_queue_node+0x79/0x143
      [<ffffffff811f2d2b>] blk_alloc_queue+0x11/0x13
      [<ffffffffa02a931d>] aoeblk_gdalloc+0x8e/0x1c9 [aoe]
      [<ffffffffa02aa655>] aoecmd_sleepwork+0x25/0xa8 [aoe]
      [<ffffffff8106186c>] worker_thread+0x1a9/0x237
      [<ffffffffa02aa630>] ? aoecmd_sleepwork+0x0/0xa8 [aoe]
      [<ffffffff81065827>] ? autoremove_wake_function+0x0/0x39
      [<ffffffff810616c3>] ? worker_thread+0x0/0x237
      [<ffffffff810653ad>] kthread+0x7f/0x87
      [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
      [<ffffffff8106532e>] ? kthread+0x0/0x87
      [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
      
      It's because there is no initialization code for a list_head contained in
      the struct backing_dev_info under CONFIG_HOTPLUG_CPU, and the bug comes up
      when block device drivers calling blk_alloc_queue() are used.  In case of
      me, I got them by using aoe.
      Signed-off-by: default avatarMasanori Itoh <itoumsn@nttdata.co.jp>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: <stable@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      8474b591
  16. 10 Aug, 2010 1 commit
    • Tim Chen's avatar
      tmpfs: add accurate compare function to percpu_counter library · 27f5e0f6
      Tim Chen authored
      Add percpu_counter_compare that allows for a quick but accurate comparison
      of percpu_counter with a given value.
      
      A rough count is provided by the count field in percpu_counter structure,
      without accounting for the other values stored in individual cpu counters.
      
      The actual count is a sum of count and the cpu counters.  However, count
      field is never different from the actual value by a factor of
      batch*num_online_cpu.  We do not need to get actual count for comparison
      if count is different from the given value by this factor and allows for
      quick comparison without summing up all the per cpu counters.
      Signed-off-by: default avatarTim Chen <tim.c.chen@linux.intel.com>
      Cc: Hugh Dickins <hughd@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      27f5e0f6
  17. 06 Jan, 2009 1 commit
  18. 29 Dec, 2008 1 commit
  19. 10 Dec, 2008 3 commits
    • Andrew Morton's avatar
      revert "percpu_counter: new function percpu_counter_sum_and_set" · 02d21168
      Andrew Morton authored
      Revert
      
          commit e8ced39d
          Author: Mingming Cao <cmm@us.ibm.com>
          Date:   Fri Jul 11 19:27:31 2008 -0400
      
              percpu_counter: new function percpu_counter_sum_and_set
      
      As described in
      
      	revert "percpu counter: clean up percpu_counter_sum_and_set()"
      
      the new percpu_counter_sum_and_set() is racy against updates to the
      cpu-local accumulators on other CPUs.  Revert that change.
      
      This means that ext4 will be slow again.  But correct.
      Reported-by: default avatarEric Dumazet <dada1@cosmosbay.com>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mingming Cao <cmm@us.ibm.com>
      Cc: <linux-ext4@vger.kernel.org>
      Cc: <stable@kernel.org>		[2.6.27.x]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      02d21168
    • Andrew Morton's avatar
      revert "percpu counter: clean up percpu_counter_sum_and_set()" · 71c5576f
      Andrew Morton authored
      Revert
      
          commit 1f7c14c6
          Author: Mingming Cao <cmm@us.ibm.com>
          Date:   Thu Oct 9 12:50:59 2008 -0400
      
              percpu counter: clean up percpu_counter_sum_and_set()
      
      Before this patch we had the following:
      
      percpu_counter_sum(): return the percpu_counter's value
      
      percpu_counter_sum_and_set(): return the percpu_counter's value, copying
      that value into the central value and zeroing the per-cpu counters before
      returning.
      
      After this patch, percpu_counter_sum_and_set() has gone, and
      percpu_counter_sum() gets the old percpu_counter_sum_and_set()
      functionality.
      
      Problem is, as Eric points out, the old percpu_counter_sum_and_set()
      functionality was racy and wrong.  It zeroes out counters on "other" cpus,
      without holding any locks which will prevent races agaist updates from
      those other CPUS.
      
      This patch reverts 1f7c14c6.  This means
      that percpu_counter_sum_and_set() still has the race, but
      percpu_counter_sum() does not.
      
      Note that this is not a simple revert - ext4 has since started using
      percpu_counter_sum() for its dirty_blocks counter as well.
      
      Note that this revert patch changes percpu_counter_sum() semantics.
      
      Before the patch, a call to percpu_counter_sum() will bring the counter's
      central counter mostly up-to-date, so a following percpu_counter_read()
      will return a close value.
      
      After this patch, a call to percpu_counter_sum() will leave the counter's
      central accumulator unaltered, so a subsequent call to
      percpu_counter_read() can now return a significantly inaccurate result.
      
      If there is any code in the tree which was introduced after
      e8ced39d was merged, and which depends
      upon the new percpu_counter_sum() semantics, that code will break.
      Reported-by: default avatarEric Dumazet <dada1@cosmosbay.com>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mingming Cao <cmm@us.ibm.com>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      71c5576f
    • Eric Dumazet's avatar
      percpu_counter: fix CPU unplug race in percpu_counter_destroy() · fd3d664f
      Eric Dumazet authored
      We should first delete the counter from percpu_counters list
      before freeing memory, or a percpu_counter_hotcpu_callback()
      could dereference a NULL pointer.
      Signed-off-by: default avatarEric Dumazet <dada1@cosmosbay.com>
      Acked-by: default avatarDavid S. Miller <davem@davemloft.net>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mingming Cao <cmm@us.ibm.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      fd3d664f
  20. 09 Oct, 2008 1 commit
  21. 11 Jul, 2008 1 commit
    • Mingming Cao's avatar
      percpu_counter: new function percpu_counter_sum_and_set · e8ced39d
      Mingming Cao authored
      Delayed allocation need to check free blocks at every write time.
      percpu_counter_read_positive() is not quit accurate. delayed
      allocation need a more accurate accounting, but using
      percpu_counter_sum_positive() is frequently is quite expensive.
      
      This patch added a new function to update center counter when sum
      per-cpu counter, to increase the accurate rate for next
      percpu_counter_read() and require less calling expensive
      percpu_counter_sum().
      Signed-off-by: default avatarMingming Cao <cmm@us.ibm.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      e8ced39d
  22. 30 Apr, 2008 1 commit
  23. 19 Oct, 2007 1 commit
  24. 17 Oct, 2007 8 commits
  25. 16 Jul, 2007 2 commits
  26. 23 Jun, 2006 2 commits