Commit aed125e2 authored by Philippe Gerum's avatar Philippe Gerum

stop_machine: ipipe: ensure atomic stop-context operations

stop_machine() guarantees that all online CPUs are spinning
non-preemptible in a known code location before a subset of them may
safely run a stop-context function. This service is typically useful
for live patching the kernel code, or changing global memory mappings,
so that no activity could run in parallel until the system has
returned to a stable state after all stop-context operations have
completed.

When interrupt pipelining is enabled, we have to provide the same
guarantee by restoring hard interrupt disabling where virtualizing the
interrupt disable flag would defeat it.
parent 3e6b6433
......@@ -449,6 +449,23 @@ extern bool force_irqthreads;
#define hard_irq_disable() do { } while(0)
#endif
/*
* Unlike other virtualized interrupt disabling schemes may assume, we
* can't expect local_irq_restore() to turn hard interrupts on when
* pipelining. hard_irq_enable() is introduced to be paired with
* hard_irq_disable(), for unconditionally turning them on. The only
* sane sequence mixing virtual and real disable state manipulation
* is:
*
* 1. local_irq_save/disable
* 2. hard_irq_disable
* 3. hard_irq_enable
* 4. local_irq_restore/enable
*/
#ifndef hard_irq_enable
#define hard_irq_enable() hard_cond_local_irq_enable()
#endif
/* PLEASE, avoid to allocate new softirqs, if you need not _really_ high
frequency threaded job scheduling. For almost all the purposes
tasklets are more than enough. F.e. all serial device BHs et
......
......@@ -144,7 +144,9 @@ static inline int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *data,
unsigned long flags;
int ret;
local_irq_save(flags);
hard_irq_disable();
ret = fn(data);
hard_irq_enable();
local_irq_restore(flags);
return ret;
}
......
......@@ -227,6 +227,7 @@ static int multi_cpu_stop(void *data)
}
} while (curstate != MULTI_STOP_EXIT);
hard_irq_enable();
local_irq_restore(flags);
return err;
}
......@@ -606,6 +607,7 @@ int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *data,
local_irq_save(flags);
hard_irq_disable();
ret = (*fn)(data);
hard_irq_enable();
local_irq_restore(flags);
return ret;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment