Traditionally, asynchronous programming in systemd has been achieved using
sd-event along with the asynchronous interfaces of sd-bus and sd-varlink.
This works well when the system is reacting to events and all code triggered
by those events can run without blocking. In these scenarios, the global
Manager object is passed as userdata to the callback, and the callback can
use the stack as usual, declaring local state and ensuring proper cleanup via
_cleanup_. Control flow structures, such as loops, work as expected, and
everything runs smoothly.
However, challenges arise when the code needs to perform long-running
operations within these callbacks. Since the system cannot block execution
within the callback, we can't directly invoke a long-running operation and
wait for its result without introducing complexities. Instead, we need to
initiate the long-running task, register for completion with sd-event,
sd-bus, or sd-varlink, and provide a callback to be invoked when the
operation completes.
This callback, however, only receives a single userdata pointer, which
forces us to bundle all local variables into a struct and pass it along as
part of the callback. On top of that, after queuing the asynchronous
operation, the caller continues executing. As the caller's stack unwinds
when the function exits, the resources and state within the local scope may
be prematurely cleaned up. Therefore, the struct must store copies of the
local variables or ensure proper reference counting to prevent premature
resource cleanup.
When multiple long-running operations need to be initiated within a loop,
the complexity grows further. We must introduce additional shared state to
track the completion of all operations before we can run any code that
depends on their results.
Furthermore, since the daemon may be shut down at any time, we must track
the lifecycle of each long-running operation in the global Manager struct,
ensuring proper cleanup even when stack unwinding can no longer manage the
resources for us.
Fibers, or green threads, provide a more natural way of handling
asynchronous operations. By enabling cooperative multitasking within a
single thread, fibers allow us to write code that looks like it’s running
synchronously, but with the ability to yield control at predefined points,
such as when waiting for long-running tasks to complete.
With fibers, we can simplify the control flow by running asynchronous
operations within a fiber, allowing us to "pause" execution while waiting
for the long-running operation to finish and then "resume" the operation once
it's complete. This eliminates the need for multiple callback chains,
extensive state tracking, and the potential pitfalls of stack unwinding.
This commit introduces the ability to execute long-running operations in a
non-blocking manner while maintaining the simplicity and readability of
synchronous code. The fiber-based approach will significantly improve the
handling of complex workflows, making the code easier to write and maintain.
The implementation is based on ucontext.h's makecontext() (with a fallback
to the venerable sigaltstack() approach on musl), sigsetjmp()/siglongjmp()
and sd-event. ucontext.h provides us with alternate stacks that we can switch
between. We use sigsetjmp()/siglongjmp() instead of swapcontext() because the
latter forcibly saves/restores a per context signal mask every time it is called.
Using sigsetjmp()/siglongjmp(), we can avoid the unnecessary syscall and maintain
a per thread signal mask, which makes much more sense than having a per fiber
signal mask.
The default stack size is the same as a regular thread. Because we
use mmap() to allocate the stack, the memory won't actually be used until it
is paged in by the kernel, so we don't actually use 8MB per fiber.
To integrate fibers with the event loop, each fiber is assigned a deferred
event source which resumes the fiber when enabled. The deferred event source
is oneshot by default so the fiber will run immediately until it yields or
suspends. If it yields, the deferred event source is enabled again (oneshot)
immediately. If it suspends, before it suspends, one or more event sources
are registered with sd-event that will enable the deferred event source
(oneshot) to resume the fiber once the operation it is waiting for completes.
Yielding or suspending the fiber is done by calling sd_fiber_yield() or
sd_fiber_suspend() respectively. Both of these return zero on success or any
error value from the async operation that caused the fiber to resume.
This is also how fiber cancellation is implemented. When a fiber is cancelled,
sd_fiber_yield() and sd_fiber_suspend() will return ECANCELED when the fiber
is resumed, allowing the fiber to unwind its stack (which allows cleanup to
happen automatically) and finish.
Instead of having applications work directly with fibers, we hide them behind
a generic futures interface to represent long-running operations, regardless of
whether those operations are running on a fiber or not. Aside from fibers, the
futures library (sd-future) will for example allow waiting for sd-event sources
and doing sd-bus calls in the background as well. Fibers can suspend until a
future is ready with sd_fiber_await() or by having the future wake up the fiber
explicitly in its callback. A future always defaults to waking up the current
fiber.
Each future kind plugs into the library by providing an sd_future_ops vtable
(alloc, free, cancel, set_priority). The library treats the impl pointer
returned by alloc() as a black box. Future Implementations retrieve it via
sd_future_get_private().
A future starts in SD_FUTURE_PENDING and transitions exactly once to
SD_FUTURE_RESOLVED, carrying an integer result. Consumers can react to that
transition either by installing a one-shot callback with
sd_future_set_callback() (callback-style code) or by waiting on it from a
fiber via sd_fiber_await() (synchronous-looking fiber code). sd_fiber_await()
is itself built on a "wait future" that resolves when its target resolves;
sd_future_new_wait() exposes the same primitive directly so non-fiber callers
can chain futures without involving a fiber.
Cancellation is cooperative: sd_future_cancel() invokes the future impl's
cancel callback, which is responsible for tearing down its work and ultimately
resolving the promise with -ECANCELED. For fiber futures this is what
surfaces as the ECANCELED return from sd_fiber_yield()/sd_fiber_suspend()
mentioned above.
Fire-and-forget fibers — created by passing a NULL ret to sd_fiber_new() —
take a self-reference on their future so they outlive the caller's scope.
The self-ref is dropped when the fiber resolves. This floating mechanism
(sd_fiber_set_floating()) is restricted to fiber futures because they
uniquely guarantee resolution; allowing it for arbitrary future kinds would
risk silent leaks for kinds that may never resolve.
Note that fiber cleanup depends on the runtime operating normally. Each
fiber's _cleanup_-style cleanups live on the fiber's own stack and run
only when the fiber is resumed and allowed to unwind, which requires a
working event loop to drive it to completion. The exit event source
registered for top-level fibers ensures unwind on a normal sd_event_exit(),
but if the event loop itself terminates abnormally (e.g. an unrecoverable
allocation failure mid-dispatch) before all fibers have resolved, their
stacks never unwind and any resources they own leak.
The code lives in libsystemd as sd-future (not exported) for the following reasons:
- We may want to make this a public libsystemd API in the future
- The code can't live in src/basic as it makes heavy use of sd-event
- The code can't live in src/shared as sd-bus and sd-event make use of it
The log and log-context headers are updated with functions to allow
fibers to have their own log prefix and log context.
Add support for filtering IRQ affinity to CPUs on a specific NUMA node
via the new IRQAffinityNUMA= option in .link files. The option accepts:
- "local": use the NUMA node local to the NIC's PCIe slot
- Explicit node number (0, 1, 2, ...): use CPUs on the specified node
When both IRQAffinity= and IRQAffinityNUMA= are specified, their
intersection is used. If the intersection is empty, an error is logged
and IRQ affinity configuration is skipped.
When "local" is specified but the device's NUMA node cannot be
determined (numa_node shows -1), a warning is logged and IRQ affinity
configuration is skipped.
Add IRQAffinity= option to .link files that filters the set of CPUs
eligible for IRQ placement. This works in conjunction with
IRQAffinityPolicy= to constrain which CPUs receive network IRQs.
When specified with spread policy, only the listed CPUs are considered
for IRQ distribution. When specified with single policy, IRQs are
pinned to the first CPU in the allowed set instead of CPU 0.
Implement the spread policy for IRQ affinity distribution using a
topology-aware algorithm. The algorithm:
1. Discovers CPU topology from sysfs (NUMA node, package, die/L3, core)
2. Groups CPUs by L3 cache domain (die) with equidistant ordering
3. Round-robins across dies, spreading IRQs across the system
4. Uses first hyperthread of each core before second hyperthreads
5. Applies IRQ affinity via /proc/irq/<n>/smp_affinity
When there are more IRQs than CPUs, queues wrap around using round-robin.
Add support for configuring IRQ affinity for network interfaces via
systemd .link files. For now, the new IRQAffinityPolicy= option in the [Link]
section only accepts "single", which pins all MSI IRQs to CPU 0.
This allows declarative IRQ affinity configuration for network devices
during udev processing, which is useful for optimizing network
performance on multi-core systems.
Further commits will expand the options supported by IRQAffinityPolicy=.
For every built executable, internal shared library, and plugin module,
verify two link-time properties via readelf:
1. No imported GLIBC symbol's version is newer than 2.34.
2. The dynamic section's NEEDED entries reference only glibc, the
runtime linker, our own libraries.
Methods:
- `io.systemd.Job.List` — list all queued jobs or look up by `id`/`unit`
name, with streaming support. Uses context/runtime split: `JobContext`
(Unit, JobType) and `JobRuntime` (Id, State, Result,
ActivationDetails). Follows the same SELinux and parameter-conflict
patterns as `io.systemd.Unit.List`.
- `io.systemd.Job.Cancel` — cancel a specific job by ID, with SELinux
and polkit authorization.
- `io.systemd.Job.ClearAll` — cancel all pending jobs, with SELinux and
polkit authorization.
Commit 881e4717c7 ("test: pin stress-ng --vm-method to a portable
scalar method in TEST-55-OOMD") pinned --vm-method=zero-one with the
rationale that it is "a long-standing scalar method". That rationale is
wrong: stress_vm_zero_one() in stress-ng's stress-vm.c is declared
static size_t TARGET_CLONES stress_vm_zero_one(...)
i.e. it carries the exact same TARGET_CLONES attribute as 33 of the 35
other vm methods. On x86_64 with GCC >=5, TARGET_CLONES expands (see
core-target-clones.h in stress-ng) to a target_clones attribute
including "arch=skylake-avx512", "arch=cooperlake", "arch=tigerlake",
"arch=sapphirerapids", and several other AVX-512-bearing arch variants,
plus "default". GCC generates AVX-512 clones of stress_vm_zero_one() and
the IFUNC resolver picks them on any CPU that advertises AVX-512.
The only vm methods in stress-ng's registry whose function definitions
omit TARGET_CLONES entirely (and are therefore guaranteed not to
dispatch to an AVX-512 clone) are lfsr32 (portable, always registered)
and write64ds (x86_64-only, gated on HAVE_ASM_X86_MOVDIRI, i.e. Intel
Tremont / Tiger Lake+ MOVDIRI instruction).
Switch the four stress-ng --vm invocations in TEST-55-OOMD to
--vm-method=lfsr32 so the AVX-512 SIGILL on CPUs without AVX-512 (e.g.
AMD Zen 1-3) can no longer occur regardless of compiler version,
optimization level, or stress-ng package build.
Follow-up for 881e4717c7
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
The test waited for the OnFailure= service's filesystem side effect
(`rmdir` of the directory) and then immediately invoked
`systemctl is-active`. Between `rmdir(2)` returning (which causes the
shell loop to exit) and PID1 reaping the child and transitioning the
oneshot service from `activating` to `active`, there is a small window
where `is-active` can observe `activating` and fail the test.
Wait directly on the unit state instead, matching the pattern used a
few lines above for the `is-failed` case.
[ 1880.326704] TEST-07-PID1.sh[21489]: + timeout --foreground 60 bash -c 'while [[ -d '\''/tmp/TEST-07-PID1-socket-8467/test'\'' ]]; do sleep .5; done'
[ 1880.330482] TEST-07-PID1.sh[21489]: + [[ ! -e /tmp/TEST-07-PID1-socket-8467/test ]]
[ 1880.330482] TEST-07-PID1.sh[21489]: + systemctl is-active TEST-07-PID1-socket-OnFailure.service
[ 1880.347470] TEST-07-PID1.sh[21520]: activating
[ 1880.349508] TEST-07-PID1.sh[21489]: + at_exit
[ 1880.349508] TEST-07-PID1.sh[21489]: + systemctl stop TEST-07-PID1-socket-8467.socket
[ 1880.367331] TEST-07-PID1.sh[107]: Subtest /usr/lib/systemd/tests/testdata/units/TEST-07-PID1.socket-on-failure.sh failed
[ 1880.367331] TEST-07-PID1.sh[107]: + return 1
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
Add COREDUMP_CODE to the fields captured by systemd-coredump. This makes
it possible for system administrators to filter coredumps based on si_code,
which describes the reason why a given signal was sent.
For example, to find processes killed due to invalid permissions
(SEGV_ACCERR):
$ journalctl COREDUMP_SIGNAL=11 COREDUMP_CODE=2
I've decided to add the value of si_code to the 'Signal: ' line of
coredumpctl info:
Signal: 11 (SEGV) si_code: SEGV_ACCERR
The stress-ng "vm" stressor's default --vm-method=all cycles through every
VM stress method, including newer ones that use AVX-512 instructions. On
CPUs without AVX-512 support (e.g. AMD Zen 1 to 3) those methods crash with
SIGILL. In testcase_oom_rulesets_lasting_sec all 10 stress-ng workers die
within ~2.34 seconds, so by the time the 6 second sleep elapses the unit
is already in failed/exit-code state and the assert_eq for
ActiveState=active trips.
Pin --vm-method=zero-one, a long-standing scalar method, on all four
stress-ng --vm invocations in this test (the two transient services in
testcase_oom_rulesets and testcase_oom_rulesets_lasting_sec, plus
TEST-55-OOMD-testbloat.service and TEST-55-OOMD-testmunch.service) so the
workers do not crash on AVX-512-less CPUs. testbloat, testmunch and
testcase_oom_rulesets have not been observed failing because they get
OOM-killed by systemd-oomd within ~1 to 2 seconds, before stress-ng cycles
into an AVX-512 method, but they share the same latent flake.
Journal excerpts from the failing run, TEST-55-OOMD-slowrule.service in
testcase_oom_rulesets_lasting_sec (journalctl -o short-monotonic):
[ 58.018676] stress-ng[1015]: invoked with '/usr/bin/stress-ng --timeout 15s --vm 10 --vm-bytes 50M --vm-keep' by user 0 'root'
[ 59.866072] stress-ng[1030]: stress-ng: debug: [1030] caught SIGILL, address 0x000055bd8d609140 (ILL_ILLOPN)
[ 59.921050] stress-ng[1030]: stress-ng: debug: [1030] stress-ng: info: 0x000055bd8d609140:<62>71 fd 48 6f 2d 36 14 1c 00 c5 d1 ef ed 49 29
[ 59.929310] stress-ng[1015]: stress-ng: error: [1015] vm: [1021] terminated with an error, exit status=2 (stressor failed)
[ 60.364111] stress-ng[1015]: stress-ng: info: [1015] failed: 10: vm (10)
[ 60.364493] stress-ng[1015]: stress-ng: info: [1015] unsuccessful run completed in 2.34 secs
[ 60.371290] systemd[1]: TEST-55-OOMD-slowrule.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
[ 60.371396] systemd[1]: TEST-55-OOMD-slowrule.service: Failed with result 'exit-code'.
[ 64.017061] TEST-55-OOMD.sh[1010]: + assert_eq failed active
[ 64.018167] TEST-55-OOMD.sh[1039]: FAIL: expected: 'active' actual: 'failed'
The faulting bytes marked by stress-ng with <62> (the byte at the
instruction pointer) decode unambiguously to an AVX-512 VMOVDQA64 using
the 512-bit zmm13 register, confirmed independently by two disassemblers:
$ printf '\x62\x71\xfd\x48\x6f\x2d\x36\x14\x1c\x00' | ndisasm -b 64 -
00000000 6271FD486F2D3614 vmovdqa64 zmm13,zword [rel 0x1c1440]
-1C00
$ echo '0x62, 0x71, 0xfd, 0x48, 0x6f, 0x2d, 0x36, 0x14, 0x1c, 0x00' | \
llvm-mc -disassemble -triple=x86_64 -mattr=+avx512f
.text
vmovdqa64 1840182(%rip), %zmm13
The leading 0x62 is the EVEX prefix (exclusive to AVX-512 on this target),
zmm13 is a 512-bit register that only exists when AVX-512 is implemented,
and VMOVDQA64 requires the AVX512F (Foundation) CPUID feature (Intel SDM
Vol 2C). Executing this on a CPU without AVX-512 raises #UD, delivered by
the kernel as SIGILL/ILL_ILLOPN, matching the journal entry above. The
same journal shows the kernel reporting "kvm_amd: TSC scaling supported",
i.e. the guest is on AMD KVM, and AMD did not ship AVX-512 before Zen 4.
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
The current key sealing format has some less-than-ideal weaknesses:
- PBKDF2 with only 65k iterations, where recommendations are ~200k
- AES with null IV, relying on salt for uniqueness
- lack of AES MAC/AEAD
However improbable, it is at least theorically possible that with
a lot of resources an offline bruteforce could be attempted.
Add a v2 sealing format, keeping unsealing compatibility with
the current format:
v2:<iterations>:<salt>:<IV>:<ciphertext>:<aes tag>
and use 600k iterations for the PBKDF2 sha512
Split the monolithic varlinkctl test script into separate files per varlink interface for better organization and easier maintenance:
- varlinkctl.sh: core varlinkctl tool tests (CLI, transports, socket discovery, upgrade/serve) and io.systemd.Manager
- varlinkctl-network.sh: io.systemd.Network
- varlinkctl-unit.sh: io.systemd.Unit (system + user manager)
- varlinkctl-metrics.sh: io.systemd.Metrics
No functional changes — the test content is moved as-is.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Introduce COREDUMP_CODE as a new captured field alongside the existing
COREDUMP_SIGNAL. While COREDUMP_SIGNAL identifies the signal number that
terminated the process, COREDUMP_CODE provides the reason the signal was sent.
For example, a process terminated by SIGSEGV due to invalid permissions would
produce COREDUMP_SIGNAL=11 and COREDUMP_CODE=2 (SEGV_ACCERR).
The kernel exposes coredump_code via pidfd starting with v7.1:
https://git.kernel.org/torvalds/c/701f7f4fbabbf4989ba6fbf033b160dd943221d5
System administrators can find both the signal and code in coredumpctl info:
$ coredumpctl info | grep Signal:
Signal: 11 (SEGV) si_code: SEGV_MAPERR
Signed-off-by: Emanuele Rocca <emanuele.rocca@arm.com>
When systemd-pull encountered a file shorter than the compression magic
headers it looks for, then it would complete the download in the
analysis state and fail.
When we are still in the analysis state and the download is done, we
know there is no compression and we should leave the analysis state and
continue writing out to disk as usual.
dhcp_message_verify_header() only enforced a lower bound on the input
length, so dhcp_message_parse() happily accepted arbitrarily large
buffers. Such inputs could never have been received via UDP and would
later fail in dhcp_message_build() with -E2BIG once the parsed options'
combined size exceeds UDP_PAYLOAD_MAX_SIZE, which the fuzzer surfaced as
an assertion failure.
Reject inputs above UDP_PAYLOAD_MAX_SIZE up front, so the parse stage
mirrors what the wire format can actually carry.
Follow-up for 8c18bb6547
Fixes https://github.com/systemd/systemd/issues/42147
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
When systemd-pull encountered a file shorter than the compression magic
headers it looks for, then it would complete the download in the
analysis state and fail.
When we are still in the analysis state and the download is done, we
know there is no compression and we should leave the analysis state and
continue writing out to disk as usual.
When SD_DHCP_OPTION_OVERLOAD indicates that the sname and/or file header
fields are overloaded with extra DHCP options, dhcp_message_parse() merges
those options into message->options but leaves the raw bytes untouched in
the header. As a result, dhcp_message_build() emits the header (including
the overloaded bytes) verbatim, and the next parse re-parses those bytes,
appending duplicate entries to the options map (each tag's iov list grows).
Subsequent builds then differ from the first, breaking the parse/build
roundtrip.
This was caught by fuzz-dhcp-client, which asserts that two consecutive
build calls produce identical output.
Zero out the overloaded fields after parsing them, since their content has
already been merged into the options map. This makes the roundtrip
idempotent and avoids re-emitting stale overloaded data in the rebuilt
header. The JSON build/parse path was already correct (it omits sname/file
from the JSON when the overload bit is set), so only the binary path needed
fixing.
Fixes https://github.com/systemd/systemd/issues/42139
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
For the shell verb we want switches specified after the program name to
be passed to the program to execute, not processed by us. Mirror the
approach in 'userdbctl ssh-authorized-keys': start with
OPTION_PARSER_RETURN_POSITIONAL_ARGS, then lates switch to
STOP_AT_FIRST_NONOPTION for "shell" or NORMAL otherwise.
VERB declarations are placed directly above each function; functions
that dispatch multiple verb names get stacked VERB() declarations.
chainload_importctl() now takes the args strv instead of relying on the
global optind.
--help output is mostly the same.
--no-pager/--no-legend/--no-ask-password/-q/--quiet are now at the end.
bind-volume/unbind-volume are documented.
Fixes c9f461a806.
Co-developed-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Daan De Meyer
Quentin Deslandes
Yu Watanabe
Luca Boccassi
Ivan Kruglov
Emanuele Rocca
Matteo Croce
Kai Lüke
Jörg Behrmann
Zbigniew Jędrzejewski-Szmek