1. 22 Jul, 2018 1 commit
  2. 20 Oct, 2016 1 commit
    • James Morse's avatar
      arm64: suspend: Reconfigure PSTATE after resume from idle · d0854412
      James Morse authored
      The suspend/resume path in kernel/sleep.S, as used by cpu-idle, does not
      save/restore PSTATE. As a result of this cpufeatures that were detected
      and have bits in PSTATE get lost when we resume from idle.
      
      UAO gets set appropriately on the next context switch. PAN will be
      re-enabled next time we return from user-space, but on a preemptible
      kernel we may run work accessing user space before this point.
      
      Add code to re-enable theses two features in __cpu_suspend_exit().
      We re-use uao_thread_switch() passing current.
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      d0854412
  3. 01 Sep, 2016 1 commit
  4. 28 Apr, 2016 2 commits
  5. 16 Feb, 2016 1 commit
  6. 17 Nov, 2015 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: pause/unpause function graph tracer in cpu_suspend() · de818bd4
      Lorenzo Pieralisi authored
      The function graph tracer adds instrumentation that is required to trace
      both entry and exit of a function. In particular the function graph
      tracer updates the "return address" of a function in order to insert
      a trace callback on function exit.
      
      Kernel power management functions like cpu_suspend() are called
      upon power down entry with functions called "finishers" that are in turn
      called to trigger the power down sequence but they may not return to the
      kernel through the normal return path.
      
      When the core resumes from low-power it returns to the cpu_suspend()
      function through the cpu_resume path, which leaves the trace stack frame
      set-up by the function tracer in an incosistent state upon return to the
      kernel when tracing is enabled.
      
      This patch fixes the issue by pausing/resuming the function graph
      tracer on the thread executing cpu_suspend() (ie the function call that
      subsequently triggers the "suspend finishers"), so that the function graph
      tracer state is kept consistent across functions that enter power down
      states and never return by effectively disabling graph tracer while they
      are executing.
      
      Fixes: 819e50e2 ("arm64: Add ftrace support")
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Reported-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Reported-by: default avatarAKASHI Takahiro <takahiro.akashi@linaro.org>
      Suggested-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      Acked-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: <stable@vger.kernel.org> # 3.16+
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      de818bd4
  7. 12 Nov, 2015 1 commit
  8. 28 Oct, 2015 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: fix tcr_el1.t0sz restore on systems with extended idmap · e13d918a
      Lorenzo Pieralisi authored
      Commit dd006da2 ("arm64: mm: increase VA range of identity map")
      introduced a mechanism to extend the virtual memory map range
      to support arm64 systems with system RAM located at very high offset,
      where the identity mapping used to enable/disable the MMU requires
      additional translation levels to map the physical memory at an equal
      virtual offset.
      
      The kernel detects at boot time the tcr_el1.t0sz value required by the
      identity mapping and sets-up the tcr_el1.t0sz register field accordingly,
      any time the identity map is required in the kernel (ie when enabling the
      MMU).
      
      After enabling the MMU, in the cold boot path the kernel resets the
      tcr_el1.t0sz to its default value (ie the actual configuration value for
      the system virtual address space) so that after enabling the MMU the
      memory space translated by ttbr0_el1 is restored as expected.
      
      Commit dd006da2 ("arm64: mm: increase VA range of identity map")
      also added code to set-up the tcr_el1.t0sz value when the kernel resumes
      from low-power states with the MMU off through cpu_resume() in order to
      effectively use the identity mapping to enable the MMU but failed to add
      the code required to restore the tcr_el1.t0sz to its default value, when
      the core returns to the kernel with the MMU enabled, so that the kernel
      might end up running with tcr_el1.t0sz value set-up for the identity
      mapping which can be lower than the value required by the actual virtual
      address space, resulting in an erroneous set-up.
      
      This patchs adds code in the resume path that restores the tcr_el1.t0sz
      default value upon core resume, mirroring this way the cold boot path
      behaviour therefore fixing the issue.
      
      Cc: <stable@vger.kernel.org>
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Fixes: dd006da2 ("arm64: mm: increase VA range of identity map")
      Acked-by: default avatarArd Biesheuvel <ard.biesheuvel@linaro.org>
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      e13d918a
  9. 07 Oct, 2015 1 commit
  10. 19 Jun, 2015 1 commit
  11. 02 Jun, 2015 1 commit
  12. 27 Jan, 2015 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: remove ARM64_CPU_SUSPEND config option · af3cfdbf
      Lorenzo Pieralisi authored
      ARM64_CPU_SUSPEND config option was introduced to make code providing
      context save/restore selectable only on platforms requiring power
      management capabilities.
      
      Currently ARM64_CPU_SUSPEND depends on the PM_SLEEP config option which
      in turn is set by the SUSPEND config option.
      
      The introduction of CPU_IDLE for arm64 requires that code configured
      by ARM64_CPU_SUSPEND (context save/restore) should be compiled in
      in order to enable the CPU idle driver to rely on CPU operations
      carrying out context save/restore.
      
      The ARM64_CPUIDLE config option (ARM64 generic idle driver) is therefore
      forced to select ARM64_CPU_SUSPEND, even if there may be (ie PM_SLEEP)
      failed dependencies, which is not a clean way of handling the kernel
      configuration option.
      
      For these reasons, this patch removes the ARM64_CPU_SUSPEND config option
      and makes the context save/restore dependent on CPU_PM, which is selected
      whenever either SUSPEND or CPU_IDLE are configured, cleaning up dependencies
      in the process.
      
      This way, code previously configured through ARM64_CPU_SUSPEND is
      compiled in whenever a power management subsystem requires it to be
      present in the kernel (SUSPEND || CPU_IDLE), which is the behaviour
      expected on ARM64 kernels.
      
      The cpu_suspend and cpu_init_idle CPU operations are added only if
      CPU_IDLE is selected, since they are CPU_IDLE specific methods and
      should be grouped and defined accordingly.
      
      PSCI CPU operations are updated to reflect the introduced changes.
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Krzysztof Kozlowski <k.kozlowski@samsung.com>
      Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      af3cfdbf
  13. 23 Dec, 2014 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: fix __cpu_suspend mm switch on warm-boot · f43c2718
      Lorenzo Pieralisi authored
      On arm64 the TTBR0_EL1 register is set to either the reserved TTBR0
      page tables on boot or to the active_mm mappings belonging to user space
      processes, it must never be set to swapper_pg_dir page tables mappings.
      
      When a CPU is booted its active_mm is set to init_mm even though its
      TTBR0_EL1 points at the reserved TTBR0 page mappings. This implies
      that when __cpu_suspend is triggered the active_mm can point at
      init_mm even if the current TTBR0_EL1 register contains the reserved
      TTBR0_EL1 mappings.
      
      Therefore, the mm save and restore executed in __cpu_suspend might
      turn out to be erroneous in that, if the current->active_mm corresponds
      to init_mm, on resume from low power it ends up restoring in the
      TTBR0_EL1 the init_mm mappings that are global and can cause speculation
      of TLB entries which end up being propagated to user space.
      
      This patch fixes the issue by checking the active_mm pointer before
      restoring the TTBR0 mappings. If the current active_mm == &init_mm,
      the code sets the TTBR0_EL1 to the reserved TTBR0 mapping instead of
      switching back to the active_mm, which is the expected behaviour
      corresponding to the TTBR0_EL1 settings when __cpu_suspend was entered.
      
      Fixes: 95322526 ("arm64: kernel: cpu_{suspend/resume} implementation")
      Cc: <stable@vger.kernel.org> # 3.14+: 18ab7db6
      Cc: <stable@vger.kernel.org> # 3.14+: 714f5992
      Cc: <stable@vger.kernel.org> # 3.14+: c3684fbb
      Cc: <stable@vger.kernel.org> # 3.14+
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      f43c2718
  14. 25 Nov, 2014 1 commit
  15. 12 Sep, 2014 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: refactor the CPU suspend API for retention states · 714f5992
      Lorenzo Pieralisi authored
      CPU suspend is the standard kernel interface to be used to enter
      low-power states on ARM64 systems. Current cpu_suspend implementation
      by default assumes that all low power states are losing the CPU context,
      so the CPU registers must be saved and cleaned to DRAM upon state
      entry. Furthermore, the current cpu_suspend() implementation assumes
      that if the CPU suspend back-end method returns when called, this has
      to be considered an error regardless of the return code (which can be
      successful) since the CPU was not expected to return from a code path that
      is different from cpu_resume code path - eg returning from the reset vector.
      
      All in all this means that the current API does not cope well with low-power
      states that preserve the CPU context when entered (ie retention states),
      since first of all the context is saved for nothing on state entry for
      those states and a successful state entry can return as a normal function
      return, which is considered an error by the current CPU suspend
      implementation.
      
      This patch refactors the cpu_suspend() API so that it can be split in
      two separate functionalities. The arm64 cpu_suspend API just provides
      a wrapper around CPU suspend operation hook. A new function is
      introduced (for architecture code use only) for states that require
      context saving upon entry:
      
      __cpu_suspend(unsigned long arg, int (*fn)(unsigned long))
      
      __cpu_suspend() saves the context on function entry and calls the
      so called suspend finisher (ie fn) to complete the suspend operation.
      The finisher is not expected to return, unless it fails in which case
      the error is propagated back to the __cpu_suspend caller.
      
      The API refactoring results in the following pseudo code call sequence for a
      suspending CPU, when triggered from a kernel subsystem:
      
      /*
       * int cpu_suspend(unsigned long idx)
       * @idx: idle state index
       */
      {
      -> cpu_suspend(idx)
      	|---> CPU operations suspend hook called, if present
      		|--> if (retention_state)
      			|--> direct suspend back-end call (eg PSCI suspend)
      		     else
      			|--> __cpu_suspend(idx, &back_end_finisher);
      }
      
      By refactoring the cpu_suspend API this way, the CPU operations back-end
      has a chance to detect whether idle states require state saving or not
      and can call the required suspend operations accordingly either through
      simple function call or indirectly through __cpu_suspend() which carries out
      state saving and suspend finisher dispatching to complete idle state entry.
      Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Reviewed-by: default avatarHanjun Guo <hanjun.guo@linaro.org>
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      714f5992
  16. 18 Jul, 2014 1 commit
  17. 24 Jan, 2014 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: fix per-cpu offset restore on resume · fb4a9602
      Lorenzo Pieralisi authored
      The introduction of percpu offset optimisation through tpidr_el1 in:
      
      Commit id :71586276
      "arm64: percpu: implement optimised pcpu access using tpidr_el1"
      
      requires cpu_{suspend/resume} to restore the tpidr_el1 register upon resume
      so that percpu variables can be addressed correctly when a CPU comes out
      of reset from warm-boot.
      
      This patch fixes cpu_{suspend}/{resume} tpidr_el1 restoration on resume, by
      calling the set_my_cpu_offset C API, as it is done on primary and secondary
      CPUs on cold boot, so that, even if the register used to store the percpu
      offset is changed, the save and restore of general purpose registers does not
      have to be updated.
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      fb4a9602
  18. 10 Jan, 2014 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: restore HW breakpoint registers in cpu_suspend · 65c021bb
      Lorenzo Pieralisi authored
      When a CPU resumes from low-power, it restores HW breakpoint and
      watchpoint slots through a CPU PM notifier. Since we want to enable
      debugging as early as possible in the resume path, the mdscr content
      is restored along the general purpose registers in the cpu_suspend API
      and debug exceptions are reenabled when cpu_suspend returns. Since the
      CPU PM notifier is run after a CPU has been resumed, we cannot expect
      HW breakpoint registers to contain sane values till the notifier is run,
      since the HW breakpoints registers content is unknown at reset; this means
      that the CPU might run with debug exceptions enabled, mdscr restored but HW
      breakpoint registers containing junk values that can trigger spurious
      debug exceptions.
      
      This patch fixes current HW breakpoints restore by moving the HW breakpoints
      registers restoration to the cpu_suspend API, before the debug exceptions are
      enabled. This way, as soon as the cpu_suspend function returns the
      kernel can resume debugging with sane values in HW breakpoint registers.
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Acked-by: default avatarWill Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      65c021bb
  19. 16 Dec, 2013 1 commit
    • Lorenzo Pieralisi's avatar
      arm64: kernel: cpu_{suspend/resume} implementation · 95322526
      Lorenzo Pieralisi authored
      Kernel subsystems like CPU idle and suspend to RAM require a generic
      mechanism to suspend a processor, save its context and put it into
      a quiescent state. The cpu_{suspend}/{resume} implementation provides
      such a framework through a kernel interface allowing to save/restore
      registers, flush the context to DRAM and suspend/resume to/from
      low-power states where processor context may be lost.
      
      The CPU suspend implementation relies on the suspend protocol registered
      in CPU operations to carry out a suspend request after context is
      saved and flushed to DRAM. The cpu_suspend interface:
      
      int cpu_suspend(unsigned long arg);
      
      allows to pass an opaque parameter that is handed over to the suspend CPU
      operations back-end so that it can take action according to the
      semantics attached to it. The arg parameter allows suspend to RAM and CPU
      idle drivers to communicate to suspend protocol back-ends; it requires
      standardization so that the interface can be reused seamlessly across
      systems, paving the way for generic drivers.
      
      Context memory is allocated on the stack, whose address is stashed in a
      per-cpu variable to keep track of it and passed to core functions that
      save/restore the registers required by the architecture.
      
      Even though, upon successful execution, the cpu_suspend function shuts
      down the suspending processor, the warm boot resume mechanism, based
      on the cpu_resume function, makes the resume path operate as a
      cpu_suspend function return, so that cpu_suspend can be treated as a C
      function by the caller, which simplifies coding the PM drivers that rely
      on the cpu_suspend API.
      
      Upon context save, the minimal amount of memory is flushed to DRAM so
      that it can be retrieved when the MMU is off and caches are not searched.
      
      The suspend CPU operation, depending on the required operations (eg CPU vs
      Cluster shutdown) is in charge of flushing the cache hierarchy either
      implicitly (by calling firmware implementations like PSCI) or explicitly
      by executing the required cache maintainance functions.
      
      Debug exceptions are disabled during cpu_{suspend}/{resume} operations
      so that debug registers can be saved and restored properly preventing
      preemption from debug agents enabled in the kernel.
      Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      95322526