• Peter Zijlstra's avatar
    perf/core: Fix sys_perf_event_open() vs. hotplug · 1c686333
    Peter Zijlstra authored
    [ Upstream commit 63cae12b ]
    There is problem with installing an event in a task that is 'stuck' on
    an offline CPU.
    Blocked tasks are not dis-assosciated from offlined CPUs, after all, a
    blocked task doesn't run and doesn't require a CPU etc.. Only on
    wakeup do we ammend the situation and place the task on a available
    If we hit such a task with perf_install_in_context() we'll loop until
    either that task wakes up or the CPU comes back online, if the task
    waking depends on the event being installed, we're stuck.
    While looking into this issue, I also spotted another problem, if we
    hit a task with perf_install_in_context() that is in the middle of
    being migrated, that is we observe the old CPU before sending the IPI,
    but run the IPI (on the old CPU) while the task is already running on
    the new CPU, things also go sideways.
    Rework things to rely on task_curr() -- outside of rq->lock -- which
    is rather tricky. Imagine the following scenario where we're trying to
    install the first event into our task 't':
    CPU0            CPU1            CPU2
                    (current == t)
    t->perf_event_ctxp[] = ctx;
    cpu = task_cpu(t);
                    switch(t, n);
                                    migrate(t, 2);
                                    switch(p, t);
                                    ctx = t->perf_event_ctxp[]; // must not be NULL
    smp_function_call(cpu, ..);
                        if (task_curr(t)) // false
                                      // sees event
    So its CPU0's store of t->perf_event_ctxp[] that must not go 'missing'.
    Because if CPU2's load of that variable were to observe NULL, it would
    not try to schedule the ctx and we'd have a task running without its
    counter, which would be 'bad'.
    As long as we observe !NULL, we'll acquire ctx->lock. If we acquire it
    first and not see the event yet, then CPU0 must observe task_curr()
    and retry. If the install happens first, then we must see the event on
    sched-in and all is well.
    I think we can translate the first part (until the 'must not be NULL')
    of the scenario to a litmus test like:
      C C-peterz
      P0(int *x, int *y)
              int r1;
              WRITE_ONCE(*x, 1);
              r1 = READ_ONCE(*y);
      P1(int *y, int *z)
              WRITE_ONCE(*y, 1);
              smp_store_release(z, 1);
      P2(int *x, int *z)
              int r1;
              int r2;
              r1 = smp_load_acquire(z);
              r2 = READ_ONCE(*x);
      (0:r1=0 /\ 2:r1=1 /\ 2:r2=0)
      x is perf_event_ctxp[],
      y is our tasks's CPU, and
      z is our task being placed on the rq of CPU2.
    The P0 smp_mb() is the one added by this patch, ordering the store to
    perf_event_ctxp[] from find_get_context() and the load of task_cpu()
    in task_function_call().
    The smp_store_release/smp_load_acquire model the RCpc locking of the
    rq->lock and the smp_mb() of P2 is the context switch switching from
    whatever CPU2 was running to our task 't'.
    This litmus test evaluates into:
      Test C-peterz Allowed
      States 7
      0:r1=0; 2:r1=0; 2:r2=0;
      0:r1=0; 2:r1=0; 2:r2=1;
      0:r1=0; 2:r1=1; 2:r2=1;
      0:r1=1; 2:r1=0; 2:r2=0;
      0:r1=1; 2:r1=0; 2:r2=1;
      0:r1=1; 2:r1=1; 2:r2=0;
      0:r1=1; 2:r1=1; 2:r2=1;
      Positive: 0 Negative: 7
      Condition exists (0:r1=0 /\ 2:r1=1 /\ 2:r2=0)
      Observation C-peterz Never 0 7
    And the strong and weak model agree.
    Reported-by: default avatarMark Rutland <mark.rutland@arm.com>
    Tested-by: default avatarMark Rutland <mark.rutland@arm.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
    Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
    Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
    Cc: Jiri Olsa <jolsa@redhat.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Stephane Eranian <eranian@google.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Vince Weaver <vincent.weaver@maine.edu>
    Cc: Will Deacon <will.deacon@arm.com>
    Cc: jeremy.linton@arm.com
    Link: http://lkml.kernel.org/r/20161209135900.GU3174@twins.programming.kicks-ass.netSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    Signed-off-by: default avatarSasha Levin <alexander.levin@verizon.com>
    Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
core.c 257 KB