When a huge number of CPUs is available (e.g. CONFIG_MAXSMP/x86), we
might overflow the stack with cpumask_t variables in
ipipe_select_timer(). Allocate the cpumask we need there dynamically
instead.

When a huge number of CPUs is available (e.g. CONFIG_MAXSMP/x86), we
might overflow the stack with cpumask_t variables in
ipipe_critical_enter().

Instead of allocating cpumask_var_t dynamically for these, rely on the
fact that we cannot reenter the code accessing them by design, so
those variables may be moved to local static storage.

Some configurations may define more than 256K distinct interrupts
(e.g. CONFIG_MAXSMP/x86), which is the limit for the current 3-level
mapping used for logging IRQs. Add a 4th mapping level to support
configurations up to 16M interrupts.

Two related issues to address:

1. Only after do_debug_exception() has told the irqsoff tracer about
the IRQ state on entry to the handler should we invoke
__ipipe_report_trap(), since the latter may have to switch the caller
to the root domain, unstalling it in the same move, which will
certainly affect the interrupt state.

2. the debug monitor handler should be called once the hardware and
virtual IRQ states are reconciled. To this end, enclose such call
inside the fault_entry/fault_exit section.

This is a pretty bad issue affecting the logic involved in reconciling
the virtual interrupt state with the hardware interrupt flags received
on entry to any CPU exception.

The net effect was a potentially corrupted virtual interrupt state on
exit from the fault handlers.

Turning on interrupts by flipping the DAIF bits is not enough when the
IRQ state is virtualized, we also have to inform the pipeline about
it, so that lockdep checks asserting IRQs on do not trigger.

Add an explicit call to local_irq_enable() prior to branching to
cpu_startup_entry(); this will do the necessary housekeeping bits such
as unstalling the root stage and telling the IRQ tracer about it.

This adds the arm64-specific bits to implement the userspace return
notifier.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Serialize accesses to the fpsimd so as to allow co-kernel activities
(e.g. tasks) running in the head domain to share the unit with the
host kernel.

The companion kernel should call fpsimd_restore_current_state() to
restore the FPU context from its own scheduling tail code.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

This commit introduces the changes redirecting "foreign" system calls
to the co-kernel.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

The mayday trap asks the co-kernel to deal with a runaway context
which has been detected earlier. When doing so, the co-kernel might
re-enable hard interrupts, which we have to disable again before
unwinding the IRQ context which triggered the whole process (like a
watchdog event).

Use __ipipe_call_mayday() to fire the notification to the co-kernel
instead of the open coded version, so that we do restore the hard
interrupt state properly before leaving __ipipe_exit_irq().
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

Notifying the co-kernel about major faults occurring in kernel context
or user-space was overlooked. We need the co-kernel to restore the
root stage whenever applicable before we may attempt to handle the
fault.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

Signed-off-by: Philippe Gerum <rpm@xenomai.org>

Raspberry 3 has two UARTs: uart0 is a full-fledged pl011 used for BT
by default, uart1 is a poor-man's low throughput serial device dubbed
as the "mini-uart". Unfortunately, uart1 is a massive pain, whose
clock rate is based on the CPU clock rate: this makes it quite
unstable as a serial console when a CPU frequency governor is enabled.

When BT is useless, we'd rather switch uart0 from BT to the serial
pins, disabling uart1 in the same move, so that we recover a decent
console device (*) with a stable clocking.

bcm2837-rpi-3-b-nobt.dtb is a fixed up blob doing exactly that.

(*) cmdline should mention console=ttyAMA0,<speed>
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

As required to mimic the original behavior when interrupt pipelining
is in effect.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

This commit introduces the changes redirecting traps and exceptions to
the interrupt pipeline, so that the co-kernel can be made aware early
on. The co-kernel may then decide whether the fault should be
propagated to the regular kernel for actual handling.

This is typically useful for allowing the co-kernel to downgrade the
current context from the head domain to the root domain, when leaving
the burden of handling major faults to the regular kernel makes more
sense than expecting the co-kernel to reinvent such a wheel
(e.g. memory violations, illegal instructions, divide by zero etc). As
a matter of fact, optimizing latency upon such events would not make
much sense anyway.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

This change enable a co-kernel to change the address space of a task
using the common mm helpers. This is mainly a matter of strictly
serializing context switching code among all callers regardless of
their domain (head/root), by disabling hard irqs.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

Context tracking does not cope well with interrupt pipelining
currently: disable the former for now.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

Introduce the arm64-specific bits enabling the interrupt pipeline
exclusively.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>

x86 may generate one, so change the signature.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Likely needed since c942cee4 which split enabling and startup.

This fixes unpopulated vectors in the IOAPIC on x86 at least, possibly
more.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

It's time to let ipipe_enable_irq return a proper error as it will gain
another function that may fail. Drop the WARN_ON_ONCE in favor of that.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Breaks in non-debug builds otherwise, e.g.
https://travis-ci.com/xenomai-ci/xenomai/jobs/212725223Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

All callers of lockdep_hardirqs_on/off already filter out !ipipe_root_p.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Lost in d7fc2c06 ("lockdep: ipipe: exclude the head stage from IRQ
state tracing") but still needed by x86 at least.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Since 9beae1ea, we are supposed to pass down flags, not just 0 or 1.
Luckily, 1 happened to be FOLL_WRITE, so we did the right thing by
chance.

Moreover, get_user_pages is deprecated in favor of its locked/unlocked/
fast variants.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

If evtdev should be NULL, we will crash on get_dev_mode(evtdev) further
down. So this test is never false in absence of sever bugs.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

A long time ago (probably in 2.6-times), someone converted spaces to
tabs, shuffling the layout around this way, and by forgetting to account
for the multi-domain removal.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Since 4.9, we need to declare continued lines via KERN_CONT.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

See https://www.xenomai.org/pipermail/xenomai/2019-March/040460.html

When a CPU is unplugged, make sure to drop all per-CPU ipipe timer
devices when removing the CPU. Otherwise, we will corrupt the device
list when re-registering the host timer on CPU onlining.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

linux/ipipe_base.h was originally designed to circumvment #include
hell by exporting only a subset of core definitions with minimum
dependencies to other inner headers.

The latest code reorganization fixed this issue in a better way, and
linux/ipipe.h is currently the only direct reader of
linux/ipipe_base.h, so let's merge both headers back as linux/ipipe.h.

At least one arch, infamous x86, has a difference of NR_syscalls
depending on compat vs. native ABI. Account for that by introducing a
function that can deliver the currently valid syscall number if an arch
implements such a service. In all other cases, this change is
functionally no difference.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

This I-pipe hook reports the desired resumption mode to the subscriber:
resume all process tasks or just single-step a particular one? The use
case is to enable synchronous stopping / resuming of all head tasks of
a ptraced real-time process.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>