1. 18 Nov, 2017 1 commit
  2. 04 Oct, 2017 2 commits
  3. 07 Sep, 2017 1 commit
    • Vitaly Wool's avatar
      z3fold: use per-cpu unbuddied lists · d30561c5
      Vitaly Wool authored
      It's been noted that z3fold doesn't scale well when it's run in a large
      number of threads on many cores, which can be easily reproduced with fio
      'randrw' test with --numjobs=32.  E.g.  the result for 1 cluster (4 cores)
      is:
      
      Run status group 0 (all jobs):
         READ: io=244785MB, aggrb=496883KB/s, minb=15527KB/s, ...
        WRITE: io=246735MB, aggrb=500841KB/s, minb=15651KB/s, ...
      
      While for 8 cores (2 clusters) the result is:
      
      Run status group 0 (all jobs):
         READ: io=244785MB, aggrb=265942KB/s, minb=8310KB/s, ...
        WRITE: io=246735MB, aggrb=268060KB/s, minb=8376KB/s, ...
      
      The bottleneck here is the pool lock which many threads become waiting
      upon.  To reduce that spin lock contention, z3fold can operate only on
      the lists local to the current CPU whenever possible.  Due to the nature
      of z3fold unbuddied list handling (it only takes the first entry off the
      list on a hot path), if the z3fold pool is big enough and balanced well
      enough, limiting search to only local unbuddied list doesn't lead to a
      significant compression ratio degrade (2.57x vs 2.65x in our
      measurements).
      
      This patch also introduces two worker threads: one for async in-page
      object layout optimization and one for releasing freed pages.  This is
      done to speed up z3fold_free() which is often on a hot path.
      
      The fio results for 8-core case are now the following:
      
      Run status group 0 (all jobs):
         READ: io=244785MB, aggrb=1568.3MB/s, minb=50182KB/s, ...
        WRITE: io=246735MB, aggrb=1580.8MB/s, minb=50582KB/s, ...
      
      So we're in for almost 6x performance increase.
      
      Link: http://lkml.kernel.org/r/20170806181443.f9b65018f8bde25ef990f9e8@gmail.comSigned-off-by: default avatarVitaly Wool <vitalywool@gmail.com>
      Cc: Dan Streetman <ddstreet@ieee.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      d30561c5
  4. 14 Apr, 2017 1 commit
    • Vitaly Wool's avatar
      z3fold: fix page locking in z3fold_alloc() · 76e32a2a
      Vitaly Wool authored
      Stress testing of the current z3fold implementation on a 8-core system
      revealed it was possible that a z3fold page deleted from its unbuddied
      list in z3fold_alloc() would be put on another unbuddied list by
      z3fold_free() while z3fold_alloc() is still processing it.  This has
      been introduced with commit 5a27aa82 ("z3fold: add kref refcounting")
      due to the removal of special handling of a z3fold page not on any list
      in z3fold_free().
      
      To fix this, the z3fold page lock should be taken in z3fold_alloc()
      before the pool lock is released.  To avoid deadlocking, we just try to
      lock the page as soon as we get a hold of it, and if trylock fails, we
      drop this page and take the next one.
      Signed-off-by: default avatarVitaly Wool <vitalywool@gmail.com>
      Cc: Dan Streetman <ddstreet@ieee.org>
      Cc: <Oleksiy.Avramchenko@sony.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      76e32a2a
  5. 16 Mar, 2017 1 commit
  6. 25 Feb, 2017 5 commits
  7. 23 Feb, 2017 1 commit
  8. 03 Jun, 2016 1 commit
  9. 21 May, 2016 1 commit