1. 09 Dec, 2017 1 commit
  2. 24 Nov, 2017 1 commit
  3. 02 Nov, 2017 1 commit
    • Greg Kroah-Hartman's avatar
      License cleanup: add SPDX GPL-2.0 license identifier to files with no license · b2441318
      Greg Kroah-Hartman authored
      Many source files in the tree are missing licensing information, which
      makes it harder for compliance tools to determine the correct license.
      
      By default all files without license information are under the default
      license of the kernel, which is GPL version 2.
      
      Update the files which contain no license information with the 'GPL-2.0'
      SPDX license identifier.  The SPDX identifier is a legally binding
      shorthand, which can be used instead of the full boiler plate text.
      
      This patch is based on work done by Thomas Gleixner and Kate Stewart and
      Philippe Ombredanne.
      
      How this work was done:
      
      Patches were generated and checked against linux-4.14-rc6 for a subset of
      the use cases:
       - file had no licensing information it it.
       - file was a */uapi/* one with no licensing information in it,
       - file was a */uapi/* one with existing licensing information,
      
      Further patches will be generated in subsequent months to fix up cases
      where non-standard license headers were used, and references to license
      had to be inferred by heuristics based on keywords.
      
      The analysis to determine which SPDX License Identifier to be applied to
      a file was done in a spreadsheet of side by side results from of the
      output of two independent scanners (ScanCode & Windriver) producing SPDX
      tag:value files created by Philippe Ombredanne.  Philippe prepared the
      base worksheet, and did an initial spot review of a few 1000 files.
      
      The 4.13 kernel was the starting point of the analysis with 60,537 files
      assessed.  Kate Stewart did a file by file comparison of the scanner
      results in the spreadsheet to determine which SPDX license identifier(s)
      to be applied to the file. She confirmed any determination that was not
      immediately clear with lawyers working with the Linux Foundation.
      
      Criteria used to select files for SPDX license identifier tagging was:
       - Files considered eligible had to be source code files.
       - Make and config files were included as candidates if they contained >5
         lines of source
       - File already had some variant of a license header in it (even if <5
         lines).
      
      All documentation files were explicitly excluded.
      
      The following heuristics were used to determine which SPDX license
      identifiers to apply.
      
       - when both scanners couldn't find any license traces, file was
         considered to have no license information in it, and the top level
         COPYING file license applied.
      
         For non */uapi/* files that summary was:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|-------
         GPL-2.0                                              11139
      
         and resulted in the first patch in this series.
      
         If that file was a */uapi/* path one, it was "GPL-2.0 WITH
         Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|-------
         GPL-2.0 WITH Linux-syscall-note                        930
      
         and resulted in the second patch in this series.
      
       - if a file had some form of licensing information in it, and was one
         of the */uapi/* ones, it was denoted with the Linux-syscall-note if
         any GPL family license was found in the file or had no licensing in
         it (per prior point).  Results summary:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|------
         GPL-2.0 WITH Linux-syscall-note                       270
         GPL-2.0+ WITH Linux-syscall-note                      169
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
         LGPL-2.1+ WITH Linux-syscall-note                      15
         GPL-1.0+ WITH Linux-syscall-note                       14
         ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
         LGPL-2.0+ WITH Linux-syscall-note                       4
         LGPL-2.1 WITH Linux-syscall-note                        3
         ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
         ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1
      
         and that resulted in the third patch in this series.
      
       - when the two scanners agreed on the detected license(s), that became
         the concluded license(s).
      
       - when there was disagreement between the two scanners (one detected a
         license but the other didn't, or they both detected different
         licenses) a manual inspection of the file occurred.
      
       - In most cases a manual inspection of the information in the file
         resulted in a clear resolution of the license that should apply (and
         which scanner probably needed to revisit its heuristics).
      
       - When it was not immediately clear, the license identifier was
         confirmed with lawyers working with the Linux Foundation.
      
       - If there was any question as to the appropriate license identifier,
         the file was flagged for further research and to be revisited later
         in time.
      
      In total, over 70 hours of logged manual review was done on the
      spreadsheet to determine the SPDX license identifiers to apply to the
      source files by Kate, Philippe, Thomas and, in some cases, confirmation
      by lawyers working with the Linux Foundation.
      
      Kate also obtained a third independent scan of the 4.13 code base from
      FOSSology, and compared selected files where the other two scanners
      disagreed against that SPDX file, to see if there was new insights.  The
      Windriver scanner is based on an older version of FOSSology in part, so
      they are related.
      
      Thomas did random spot checks in about 500 files from the spreadsheets
      for the uapi headers and agreed with SPDX license identifier in the
      files he inspected. For the non-uapi files Thomas did random spot checks
      in about 15000 files.
      
      In initial set of patches against 4.14-rc6, 3 files were found to have
      copy/paste license identifier errors, and have been fixed to reflect the
      correct identifier.
      
      Additionally Philippe spent 10 hours this week doing a detailed manual
      inspection and review of the 12,461 patched files from the initial patch
      version early this week with:
       - a full scancode scan run, collecting the matched texts, detected
         license ids and scores
       - reviewing anything where there was a license detected (about 500+
         files) to ensure that the applied SPDX license was correct
       - reviewing anything where there was no detection but the patch license
         was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
         SPDX license was correct
      
      This produced a worksheet with 20 files needing minor correction.  This
      worksheet was then exported into 3 different .csv files for the
      different types of files to be modified.
      
      These .csv files were then reviewed by Greg.  Thomas wrote a script to
      parse the csv files and add the proper SPDX tag to the file, in the
      format that the file expected.  This script was further refined by Greg
      based on the output to detect more types of files automatically and to
      distinguish between header and source .c files (which need different
      comment types.)  Finally Greg ran the script using the .csv files to
      generate the patches.
      Reviewed-by: default avatarKate Stewart <kstewart@linuxfoundation.org>
      Reviewed-by: default avatarPhilippe Ombredanne <pombredanne@nexb.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      b2441318
  4. 20 Oct, 2017 1 commit
  5. 03 Oct, 2017 1 commit
  6. 23 Sep, 2017 1 commit
    • Paul E. McKenney's avatar
      rcu: Allow for page faults in NMI handlers · 28585a83
      Paul E. McKenney authored
      A number of architecture invoke rcu_irq_enter() on exception entry in
      order to allow RCU read-side critical sections in the exception handler
      when the exception is from an idle or nohz_full CPU.  This works, at
      least unless the exception happens in an NMI handler.  In that case,
      rcu_nmi_enter() would already have exited the extended quiescent state,
      which would mean that rcu_irq_enter() would (incorrectly) cause RCU
      to think that it is again in an extended quiescent state.  This will
      in turn result in lockdep splats in response to later RCU read-side
      critical sections.
      
      This commit therefore causes rcu_irq_enter() and rcu_irq_exit() to
      take no action if there is an rcu_nmi_enter() in effect, thus avoiding
      the unscheduled return to RCU quiescent state.  This in turn should
      make the kernel safe for on-demand RCU voyeurism.
      
      Link: http://lkml.kernel.org/r/20170922211022.GA18084@linux.vnet.ibm.com
      
      Cc: stable@vger.kernel.org
      Fixes: 0be964be ("module: Sanitize RCU usage and locking")
      Reported-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Signed-off-by: default avatarSteven Rostedt (VMware) <rostedt@goodmis.org>
      28585a83
  7. 09 Sep, 2017 1 commit
  8. 17 Aug, 2017 12 commits
  9. 27 Jul, 2017 1 commit
    • Paul E. McKenney's avatar
      srcu: Provide ordering for CPU not involved in grace period · 35732cf9
      Paul E. McKenney authored
      Tree RCU guarantees that every online CPU has a memory barrier between
      any given grace period and any of that CPU's RCU read-side sections that
      must be ordered against that grace period.  Since RCU doesn't always
      know where read-side critical sections are, the actual implementation
      guarantees order against prior and subsequent non-idle non-offline code,
      whether in an RCU read-side critical section or not.  As a result, there
      does not need to be a memory barrier at the end of synchronize_rcu()
      and friends because the ordering internal to the grace period has
      ordered every CPU's post-grace-period execution against each CPU's
      pre-grace-period execution, again for all non-idle online CPUs.
      
      In contrast, SRCU can have non-idle online CPUs that are completely
      uninvolved in a given SRCU grace period, for example, a CPU that
      never runs any SRCU read-side critical sections and took no part in
      the grace-period processing.  It is in theory possible for a given
      synchronize_srcu()'s wakeup to be delivered to a CPU that was completely
      uninvolved in the prior SRCU grace period, which could mean that the
      code following that synchronize_srcu() would end up being unordered with
      respect to both the grace period and any pre-existing SRCU read-side
      critical sections.
      
      This commit therefore adds an smp_mb() to the end of __synchronize_srcu(),
      which prevents this scenario from occurring.
      Reported-by: default avatarLance Roy <ldr709@gmail.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Acked-by: default avatarLance Roy <ldr709@gmail.com>
      Cc: <stable@vger.kernel.org> # 4.12.x
      35732cf9
  10. 25 Jul, 2017 12 commits
    • Paul E. McKenney's avatar
      rcu: Move callback-list warning to irq-disable region · 09efeeee
      Paul E. McKenney authored
      After adopting callbacks from a newly offlined CPU, the adopting CPU
      checks to make sure that its callback list's count is zero only if the
      list has no callbacks and vice versa.  Unfortunately, it does so after
      enabling interrupts, which means that false positives are possible due to
      interrupt handlers invoking call_rcu().  Although these false positives
      are improbable, rcutorture did make it happen once.
      
      This commit therefore moves this check to an irq-disabled region of code,
      thus suppressing the false positive.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      09efeeee
    • Paul E. McKenney's avatar
      rcu: Remove unused RCU list functions · aed4e046
      Paul E. McKenney authored
      Given changes to callback migration, rcu_cblist_head(),
      rcu_cblist_tail(), rcu_cblist_count_cbs(), rcu_segcblist_segempty(),
      rcu_segcblist_dequeued_lazy(), and rcu_segcblist_new_cbs() are
      no longer used.  This commit therefore removes them.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      aed4e046
    • Paul E. McKenney's avatar
      rcu: Localize rcu_state ->orphan_pend and ->orphan_done · f2dbe4a5
      Paul E. McKenney authored
      Given that the rcu_state structure's >orphan_pend and ->orphan_done
      fields are used only during migration of callbacks from the recently
      offlined CPU to a surviving CPU, if rcu_send_cbs_to_orphanage() and
      rcu_adopt_orphan_cbs() are combined, these fields can become local
      variables in the combined function.  This commit therefore combines
      rcu_send_cbs_to_orphanage() and rcu_adopt_orphan_cbs() into a new
      rcu_segcblist_merge() function and removes the ->orphan_pend and
      ->orphan_done fields.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      f2dbe4a5
    • Paul E. McKenney's avatar
      rcu: Advance callbacks after migration · 21cc2483
      Paul E. McKenney authored
      When migrating callbacks from a newly offlined CPU, we are already
      holding the root rcu_node structure's lock, so it costs almost nothing
      to advance and accelerate the newly migrated callbacks.  This patch
      therefore makes this advancing and acceleration happen.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      21cc2483
    • Paul E. McKenney's avatar
      rcu: Eliminate rcu_state ->orphan_lock · 537b85c8
      Paul E. McKenney authored
      The ->orphan_lock is acquired and released only within the
      rcu_migrate_callbacks() function, which now acquires the root rcu_node
      structure's ->lock.  This commit therefore eliminates the ->orphan_lock
      in favor of the root rcu_node structure's ->lock.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      537b85c8
    • Paul E. McKenney's avatar
      rcu: Advance outgoing CPU's callbacks before migrating them · 9fa46fb8
      Paul E. McKenney authored
      It is possible that the outgoing CPU is unaware of recent grace periods,
      and so it is also possible that some of its pending callbacks are actually
      ready to be invoked.  The current callback-migration code would needlessly
      force these callbacks to pass through another grace period.  This commit
      therefore invokes rcu_advance_cbs() on the outgoing CPU's callbacks in
      order to give them full credit for having passed through any recent
      grace periods.
      
      This also fixes an odd theoretical bug where there are no callbacks in
      the system except for those on the outgoing CPU, none of those callbacks
      have yet been associated with a grace-period number, there is never again
      another callback registered, and the surviving CPU never again takes a
      scheduling-clock interrupt, never goes idle, and never enters nohz_full
      userspace execution.  Yes, this is (just barely) possible.  It requires
      that the surviving CPU be a nohz_full CPU, that its scheduler-clock
      interrupt be shut off, and that it loop forever in the kernel.  You get
      bonus points if you can make this one happen!  ;-)
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      9fa46fb8
    • Paul E. McKenney's avatar
      rcu: Make NOCB CPUs migrate CBs directly from outgoing CPU · b1a2d79f
      Paul E. McKenney authored
      RCU's CPU-hotplug callback-migration code first moves the outgoing
      CPU's callbacks to ->orphan_done and ->orphan_pend, and only then
      moves them to the NOCB callback list.  This commit avoids the
      extra step (and simplifies the code) by moving the callbacks directly
      from the outgoing CPU's callback list to the NOCB callback list.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      b1a2d79f
    • Paul E. McKenney's avatar
      rcu: Check for NOCB CPUs and empty lists earlier in CB migration · 95335c03
      Paul E. McKenney authored
      The current CPU-hotplug RCU-callback-migration code checks
      for the source (newly offlined) CPU being a NOCBs CPU down in
      rcu_send_cbs_to_orphanage().  This commit simplifies callback migration a
      bit by moving this check up to rcu_migrate_callbacks().  This commit also
      adds a check for the source CPU having no callbacks, which eases analysis
      of the rcu_send_cbs_to_orphanage() and rcu_adopt_orphan_cbs() functions.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      95335c03
    • Paul E. McKenney's avatar
      rcu: Remove orphan/adopt event-tracing fields · c47e067a
      Paul E. McKenney authored
      The rcu_node structure's ->n_cbs_orphaned and ->n_cbs_adopted fields
      are updated, but never read.  This commit therefore removes them.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      c47e067a
    • Paul E. McKenney's avatar
      rcu: Make expedited GPs correctly handle hardware CPU insertion · 313517fc
      Paul E. McKenney authored
      The update of the ->expmaskinitnext and of ->ncpus are unsynchronized,
      with the value of ->ncpus being incremented long before the corresponding
      ->expmaskinitnext mask is updated.  If an RCU expedited grace period
      sees ->ncpus change, it will update the ->expmaskinit masks from the new
      ->expmaskinitnext masks.  But it is possible that ->ncpus has already
      been updated, but the ->expmaskinitnext masks still have their old values.
      For the current expedited grace period, no harm done.  The CPU could not
      have been online before the grace period started, so there is no need to
      wait for its non-existent pre-existing readers.
      
      But the next RCU expedited grace period is in a world of hurt.  The value
      of ->ncpus has already been updated, so this grace period will assume
      that the ->expmaskinitnext masks have not changed.  But they have, and
      they won't be taken into account until the next never-been-online CPU
      comes online.  This means that RCU will be ignoring some CPUs that it
      should be paying attention to.
      
      The solution is to update ->ncpus and ->expmaskinitnext while holding
      the ->lock for the rcu_node structure containing the ->expmaskinitnext
      mask.  Because smp_store_release() is now used to update ->ncpus and
      smp_load_acquire() is now used to locklessly read it, if the expedited
      grace period sees ->ncpus change, then the updating CPU has to
      already be holding the corresponding ->lock.  Therefore, when the
      expedited grace period later acquires that ->lock, it is guaranteed
      to see the new value of ->expmaskinitnext.
      
      On the other hand, if the expedited grace period loads ->ncpus just
      before an update, earlier full memory barriers guarantee that
      the incoming CPU isn't far enough along to be running any RCU readers.
      
      This commit therefore makes the required change.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      313517fc
    • Paul E. McKenney's avatar
      rcu: Migrate callbacks earlier in the CPU-offline timeline · a58163d8
      Paul E. McKenney authored
      RCU callbacks must be migrated away from an outgoing CPU, and this is
      done near the end of the CPU-hotplug operation, after the outgoing CPU is
      long gone.  Unfortunately, this means that other CPU-hotplug callbacks
      can execute while the outgoing CPU's callbacks are still immobilized
      on the long-gone CPU's callback lists.  If any of these CPU-hotplug
      callbacks must wait, either directly or indirectly, for the invocation
      of any of the immobilized RCU callbacks, the system will hang.
      
      This commit avoids such hangs by migrating the callbacks away from the
      outgoing CPU immediately upon its departure, shortly after the return
      from __cpu_die() in takedown_cpu().  Thus, RCU is able to advance these
      callbacks and invoke them, which allows all the after-the-fact CPU-hotplug
      callbacks to wait on these RCU callbacks without risk of a hang.
      
      While in the neighborhood, this commit also moves rcu_send_cbs_to_orphanage()
      and rcu_adopt_orphan_cbs() under a pre-existing #ifdef to avoid including
      dead code on the one hand and to avoid define-without-use warnings on the
      other hand.
      Reported-by: default avatarJeffrey Hugo <jhugo@codeaurora.org>
      Link: http://lkml.kernel.org/r/db9c91f6-1b17-6136-84f0-03c3c2581ab4@codeaurora.orgSigned-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Anna-Maria Gleixner <anna-maria@linutronix.de>
      Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
      Cc: Richard Weinberger <richard@nod.at>
      a58163d8
    • Paul E. McKenney's avatar
      rcu: Use timer as backstop for NOCB deferred wakeups · 8be6e1b1
      Paul E. McKenney authored
      The handling of RCU's no-CBs CPUs has a maintenance headache, namely
      that if call_rcu() is invoked with interrupts disabled, the rcuo kthread
      wakeup must be defered to a point where we can be sure that scheduler
      locks are not held.  Of course, there are a lot of code paths leading
      from an interrupts-disabled invocation of call_rcu(), and missing any
      one of these can result in excessive callback-invocation latency, and
      potentially even system hangs.
      
      This commit therefore uses a timer to guarantee that the wakeup will
      eventually occur.  If one of the deferred-wakeup points kicks in, then
      the timer is simply cancelled.
      
      This commit also fixes up an incomplete removal of commits that were
      intended to plug remaining exit paths, which should have the added
      benefit of reducing the overhead of RCU's context-switch hooks.  In
      addition, it simplifies leader-to-follower callback-list handoff by
      introducing locking.  The call_rcu()-to-leader handoff continues to
      use atomic operations in order to maintain good real-time latency for
      common-case use of call_rcu().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      [ paulmck: Dan Carpenter fix for mod_timer() usage bug found by smatch. ]
      8be6e1b1
  11. 24 Jul, 2017 8 commits