Traditionally, asynchronous programming in systemd has been achieved
using
sd-event along with the asynchronous interfaces of sd-bus and
sd-varlink.
This works well when the system is reacting to events and all code
triggered
by those events can run without blocking. In these scenarios, the global
Manager object is passed as userdata to the callback, and the callback
can
use the stack as usual, declaring local state and ensuring proper
cleanup via
_cleanup_. Control flow structures, such as loops, work as expected, and
everything runs smoothly.
However, challenges arise when the code needs to perform long-running
operations within these callbacks. Since the system cannot block
execution
within the callback, we can't directly invoke a long-running operation
and
wait for its result without introducing complexities. Instead, we need
to
initiate the long-running task, register for completion with sd-event,
sd-bus, or sd-varlink, and provide a callback to be invoked when the
operation completes.
This callback, however, only receives a single userdata pointer, which
forces us to bundle all local variables into a struct and pass it along
as
part of the callback. On top of that, after queuing the asynchronous
operation, the caller continues executing. As the caller's stack unwinds
when the function exits, the resources and state within the local scope
may
be prematurely cleaned up. Therefore, the struct must store copies of
the
local variables or ensure proper reference counting to prevent premature
resource cleanup.
When multiple long-running operations need to be initiated within a
loop,
the complexity grows further. We must introduce additional shared state
to
track the completion of all operations before we can run any code that
depends on their results.
Furthermore, since the daemon may be shut down at any time, we must
track
the lifecycle of each long-running operation in the global Manager
struct,
ensuring proper cleanup even when stack unwinding can no longer manage
the
resources for us.
Fibers, or green threads, provide a more natural way of handling
asynchronous operations. By enabling cooperative multitasking within a
single thread, fibers allow us to write code that looks like it’s
running
synchronously, but with the ability to yield control at predefined
points,
such as when waiting for long-running tasks to complete.
With fibers, we can simplify the control flow by running asynchronous
operations within a fiber, allowing us to "pause" execution while
waiting
for the long-running operation to finish and then "resume" the operation
once
it's complete. This eliminates the need for multiple callback chains,
extensive state tracking, and the potential pitfalls of stack unwinding.
This commit introduces the ability to execute long-running operations in
a
non-blocking manner while maintaining the simplicity and readability of
synchronous code. The fiber-based approach will significantly improve
the
handling of complex workflows, making the code easier to write and
maintain.
The implementation is based on ucontext.h's makecontext() (with a
fallback
to the venerable sigaltstack() approach on musl),
sigsetjmp()/siglongjmp()
and sd-event. ucontext.h provides us with alternate stacks that we can
switch
between. We use sigsetjmp()/siglongjmp() instead of swapcontext()
because the
latter forcibly saves/restores a per context signal mask every time it
is called.
Using sigsetjmp()/siglongjmp(), we can avoid the unnecessary syscall and
maintain
a per thread signal mask, which makes much more sense than having a per
fiber
signal mask.
The default stack size is the same as a regular thread. Because we
use mmap() to allocate the stack, the memory won't actually be used
until it
is paged in by the kernel, so we don't actually use 8MB per fiber.
To integrate fibers with the event loop, each fiber is assigned a
deferred
event source which resumes the fiber when enabled. The deferred event
source
is oneshot by default so the fiber will run immediately until it yields
or
suspends. If it yields, the deferred event source is enabled again
(oneshot)
immediately. If it suspends, before it suspends, one or more event
sources
are registered with sd-event that will enable the deferred event source
(oneshot) to resume the fiber once the operation it is waiting for
completes.
Yielding or suspending the fiber is done by calling sd_fiber_yield() or
sd_fiber_suspend() respectively. Both of these return zero on success or
any
error value from the async operation that caused the fiber to resume.
This is also how fiber cancellation is implemented. When a fiber is
cancelled,
sd_fiber_yield() and sd_fiber_suspend() will return ECANCELED when the
fiber
is resumed, allowing the fiber to unwind its stack (which allows cleanup
to
happen automatically) and finish.
Instead of having applications work directly with fibers, we hide them
behind
a generic futures interface to represent long-running operations,
regardless of
whether those operations are running on a fiber or not. Aside from
fibers, the
futures library (sd-future) will for example allow waiting for sd-event
sources
and doing sd-bus calls in the background as well. Fibers can suspend
until a
future is ready with sd_fiber_await() or by having the future wake up
the fiber
explicitly in its callback. A future always defaults to waking up the
current
fiber.
Each future kind plugs into the library by providing an sd_future_ops
vtable
(alloc, free, cancel, set_priority). The library treats the impl pointer
returned by alloc() as a black box. Future Implementations retrieve it
via
sd_future_get_private().
A future starts in SD_FUTURE_PENDING and transitions exactly once to
SD_FUTURE_RESOLVED, carrying an integer result. Consumers can react to
that
transition either by installing a one-shot callback with
sd_future_set_callback() (callback-style code) or by waiting on it from
a
fiber via sd_fiber_await() (synchronous-looking fiber code).
sd_fiber_await()
is itself built on a "wait future" that resolves when its target
resolves;
sd_future_new_wait() exposes the same primitive directly so non-fiber
callers
can chain futures without involving a fiber.
Cancellation is cooperative: sd_future_cancel() invokes the future
impl's
cancel callback, which is responsible for tearing down its work and
ultimately
resolving the promise with -ECANCELED. For fiber futures this is what
surfaces as the ECANCELED return from
sd_fiber_yield()/sd_fiber_suspend()
mentioned above.
Fire-and-forget fibers — created by passing a NULL ret to sd_fiber_new()
—
take a self-reference on their future so they outlive the caller's
scope.
The self-ref is dropped when the fiber resolves. This floating mechanism
(sd_fiber_set_floating()) is restricted to fiber futures because they
uniquely guarantee resolution; allowing it for arbitrary future kinds
would
risk silent leaks for kinds that may never resolve.
Note that fiber cleanup depends on the runtime operating normally. Each
fiber's _cleanup_-style cleanups live on the fiber's own stack and run
only when the fiber is resumed and allowed to unwind, which requires a
working event loop to drive it to completion. The exit event source
registered for top-level fibers ensures unwind on a normal
sd_event_exit(),
but if the event loop itself terminates abnormally (e.g. an
unrecoverable
allocation failure mid-dispatch) before all fibers have resolved, their
stacks never unwind and any resources they own leak.
The code lives in libsystemd as sd-future (not exported) for the
following reasons:
- We may want to make this a public libsystemd API in the future
- The code can't live in src/basic as it makes heavy use of sd-event
- The code can't live in src/shared as sd-bus and sd-event make use of
it
The log and log-context headers are updated with functions to allow
fibers to have their own log prefix and log context.
Add -fexcess-precision=standard so gcc inserts ISO C99 conformant
rounding at assignments, casts, and returns — without it, double values
on x87 happily stay at 80-bit extended precision across operations and
diverge from the SSE/x86_64 behavior, making strict equality comparisons
architecture-dependent.
The flag doesn't fully cover x87: per gcc PR#323
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=323), a function return
value carried in ST(0) can arrive at the caller still at 80-bit, so a
double that ought to compare equal to a same-magnitude literal picks up
extra mantissa bits and doesn't. Wrap fp_equal in volatile-double
temporaries to force a memory roundtrip — the only operation that
reliably truncates on x87 — so its callers get consistent results
regardless of how the operands were produced.
Add a TEST(fp_equal) case that exercises the previously-broken pattern:
a runtime 1.0/10.0 computed inside a noinline helper, returned across
the function ABI boundary, then compared against the literal 0.1.
Without the volatile truncation this assertion fails on 32-bit gcc.
Traditionally, asynchronous programming in systemd has been achieved using
sd-event along with the asynchronous interfaces of sd-bus and sd-varlink.
This works well when the system is reacting to events and all code triggered
by those events can run without blocking. In these scenarios, the global
Manager object is passed as userdata to the callback, and the callback can
use the stack as usual, declaring local state and ensuring proper cleanup via
_cleanup_. Control flow structures, such as loops, work as expected, and
everything runs smoothly.
However, challenges arise when the code needs to perform long-running
operations within these callbacks. Since the system cannot block execution
within the callback, we can't directly invoke a long-running operation and
wait for its result without introducing complexities. Instead, we need to
initiate the long-running task, register for completion with sd-event,
sd-bus, or sd-varlink, and provide a callback to be invoked when the
operation completes.
This callback, however, only receives a single userdata pointer, which
forces us to bundle all local variables into a struct and pass it along as
part of the callback. On top of that, after queuing the asynchronous
operation, the caller continues executing. As the caller's stack unwinds
when the function exits, the resources and state within the local scope may
be prematurely cleaned up. Therefore, the struct must store copies of the
local variables or ensure proper reference counting to prevent premature
resource cleanup.
When multiple long-running operations need to be initiated within a loop,
the complexity grows further. We must introduce additional shared state to
track the completion of all operations before we can run any code that
depends on their results.
Furthermore, since the daemon may be shut down at any time, we must track
the lifecycle of each long-running operation in the global Manager struct,
ensuring proper cleanup even when stack unwinding can no longer manage the
resources for us.
Fibers, or green threads, provide a more natural way of handling
asynchronous operations. By enabling cooperative multitasking within a
single thread, fibers allow us to write code that looks like it’s running
synchronously, but with the ability to yield control at predefined points,
such as when waiting for long-running tasks to complete.
With fibers, we can simplify the control flow by running asynchronous
operations within a fiber, allowing us to "pause" execution while waiting
for the long-running operation to finish and then "resume" the operation once
it's complete. This eliminates the need for multiple callback chains,
extensive state tracking, and the potential pitfalls of stack unwinding.
This commit introduces the ability to execute long-running operations in a
non-blocking manner while maintaining the simplicity and readability of
synchronous code. The fiber-based approach will significantly improve the
handling of complex workflows, making the code easier to write and maintain.
The implementation is based on ucontext.h's makecontext() (with a fallback
to the venerable sigaltstack() approach on musl), sigsetjmp()/siglongjmp()
and sd-event. ucontext.h provides us with alternate stacks that we can switch
between. We use sigsetjmp()/siglongjmp() instead of swapcontext() because the
latter forcibly saves/restores a per context signal mask every time it is called.
Using sigsetjmp()/siglongjmp(), we can avoid the unnecessary syscall and maintain
a per thread signal mask, which makes much more sense than having a per fiber
signal mask.
The default stack size is the same as a regular thread. Because we
use mmap() to allocate the stack, the memory won't actually be used until it
is paged in by the kernel, so we don't actually use 8MB per fiber.
To integrate fibers with the event loop, each fiber is assigned a deferred
event source which resumes the fiber when enabled. The deferred event source
is oneshot by default so the fiber will run immediately until it yields or
suspends. If it yields, the deferred event source is enabled again (oneshot)
immediately. If it suspends, before it suspends, one or more event sources
are registered with sd-event that will enable the deferred event source
(oneshot) to resume the fiber once the operation it is waiting for completes.
Yielding or suspending the fiber is done by calling sd_fiber_yield() or
sd_fiber_suspend() respectively. Both of these return zero on success or any
error value from the async operation that caused the fiber to resume.
This is also how fiber cancellation is implemented. When a fiber is cancelled,
sd_fiber_yield() and sd_fiber_suspend() will return ECANCELED when the fiber
is resumed, allowing the fiber to unwind its stack (which allows cleanup to
happen automatically) and finish.
Instead of having applications work directly with fibers, we hide them behind
a generic futures interface to represent long-running operations, regardless of
whether those operations are running on a fiber or not. Aside from fibers, the
futures library (sd-future) will for example allow waiting for sd-event sources
and doing sd-bus calls in the background as well. Fibers can suspend until a
future is ready with sd_fiber_await() or by having the future wake up the fiber
explicitly in its callback. A future always defaults to waking up the current
fiber.
Each future kind plugs into the library by providing an sd_future_ops vtable
(alloc, free, cancel, set_priority). The library treats the impl pointer
returned by alloc() as a black box. Future Implementations retrieve it via
sd_future_get_private().
A future starts in SD_FUTURE_PENDING and transitions exactly once to
SD_FUTURE_RESOLVED, carrying an integer result. Consumers can react to that
transition either by installing a one-shot callback with
sd_future_set_callback() (callback-style code) or by waiting on it from a
fiber via sd_fiber_await() (synchronous-looking fiber code). sd_fiber_await()
is itself built on a "wait future" that resolves when its target resolves;
sd_future_new_wait() exposes the same primitive directly so non-fiber callers
can chain futures without involving a fiber.
Cancellation is cooperative: sd_future_cancel() invokes the future impl's
cancel callback, which is responsible for tearing down its work and ultimately
resolving the promise with -ECANCELED. For fiber futures this is what
surfaces as the ECANCELED return from sd_fiber_yield()/sd_fiber_suspend()
mentioned above.
Fire-and-forget fibers — created by passing a NULL ret to sd_fiber_new() —
take a self-reference on their future so they outlive the caller's scope.
The self-ref is dropped when the fiber resolves. This floating mechanism
(sd_fiber_set_floating()) is restricted to fiber futures because they
uniquely guarantee resolution; allowing it for arbitrary future kinds would
risk silent leaks for kinds that may never resolve.
Note that fiber cleanup depends on the runtime operating normally. Each
fiber's _cleanup_-style cleanups live on the fiber's own stack and run
only when the fiber is resumed and allowed to unwind, which requires a
working event loop to drive it to completion. The exit event source
registered for top-level fibers ensures unwind on a normal sd_event_exit(),
but if the event loop itself terminates abnormally (e.g. an unrecoverable
allocation failure mid-dispatch) before all fibers have resolved, their
stacks never unwind and any resources they own leak.
The code lives in libsystemd as sd-future (not exported) for the following reasons:
- We may want to make this a public libsystemd API in the future
- The code can't live in src/basic as it makes heavy use of sd-event
- The code can't live in src/shared as sd-bus and sd-event make use of it
The log and log-context headers are updated with functions to allow
fibers to have their own log prefix and log context.
Drop the -fundamental suffix from src/fundamental/ headers in favor of names
that match their src/basic/ or src/shared/ counterparts (e.g.
macro-fundamental.h -> macro.h, assert-fundamental.h -> assert-util.h,
cleanup-fundamental.h -> cleanup-util.h). Rename src/basic/{btrfs,label}.{c,h}
to use the -util suffix to match the existing shared/btrfs-util and
shared/label-util siblings. Rename src/shared/mkdir-label.{c,h} to mkdir.{c,h}
and src/shared/tmpfile-util-label.{c,h} to tmpfile-util.{c,h} to match the
corresponding src/basic names.
This saves us from having to come up with separate names for files that do
the same thing across tiers, and it makes it easier to move stuff between
src/fundamental, src/basic and src/shared: consumers just #include "foo.h"
and pick up whichever tier their -I path resolves to first, so call sites
don't need to be updated when an API moves between layers.
Where a higher-tier wrapper exists (e.g. src/basic/macro.h wrapping
src/fundamental/macro.h), the wrapper uses an explicit "../fundamental/foo.h"
or "../basic/foo.h" relative include for the lower-tier header. We can't use
GCC's #include_next directive for this — when the wrapper is reachable both
via same-dir-as-source lookup and via -I (e.g. -Isrc/shared) for the
directory it lives in, #include_next advances by exactly one slot in libcpp's
internal directory chain and lands on the same physical directory it was
already in, never reaching the lower-tier sibling (see make_cpp_dir() in
gcc/libcpp/files.cc:1986).
To make sure the right headers are always picked up, the include directories
are reordered so that e.g. src/shared always takes priority over src/basic and
similar for the other directories.
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
- test-random-util is reworked to not use sqrt()
- pretty-print.c inlines ceil() so libm doesn't have
to be linked into libshared
- We add fno-math-errno to allow inlining of more math
functions by not requiring standard math functions to
set errno on invalid input.
For every built executable, internal shared library, and plugin module,
verify two link-time properties via readelf:
1. No imported GLIBC symbol's version is newer than 2.34.
2. The dynamic section's NEEDED entries reference only glibc, the
runtime linker, our own libraries.
meson test --wrapper hook to print a gdb backtrace inline in the test
log when a test exits with an actual crash signal (SIGSEGV, SIGABRT,
SIGBUS, SIGFPE, SIGILL). Wired into the default add_test_setup() so it
runs automatically on every `meson test`.
Environmental terminations (SIGTERM/SIGKILL/SIGPIPE/SIGALRM) are passed
through without replay, and the original signal is re-raised so the
parent's wait() observes WIFSIGNALED rather than a plain exit code.
- **bpf-util: rename from bpf-dlopen, unify version-specific symbol
handling**
- **cryptsetup: dlopen libcryptsetup in tokens**
- **tree-wide: dlopen libpam in pam plugins**
- **test-bus-marshal: dlopen() glib and libdbus instead of linking
directly**
- **lock-util: Simplify timeout for lock_generic_with_timeout()**
- **color-util: simplify hsv_to_rgb, fix rgb_to_hsv negative-hue wrap**
- **tree-wide: Use our own macros instead of fabs()/fmax()/fmin()**
- **locale-util: dlopen() libintl instead of linking against it**
- **home: Use log2u64() over log2()**
- **meson: drop libdl, threads, and librt dependencies**
- **libc: Use dlsym() from a constructor instead of weak symbols**
- **libc: Make sure C23 versions of strtol(), sscanf() are not used**
When _GNU_SOURCE is defined, glibc will always use c23 versions
of strtol(), sscanf() and friends if available (introduced after
glibc 2.34). Which means that any binaries built with headers
from newer glibc won't load on glibc < 2.38. To work around this,
redefine the appropriate constants to zero make sure the c99
versions are used instead.
Our baseline glibc is 2.34, which merged libdl, libpthread (the
dependency('threads') target), and librt into libc. Empty .so/.a stubs
remain for backward compatibility with old binaries, but new builds
resolve dl_*, pthread_*, mq_*, timer_*, etc. directly from libc.
On musl the same libraries are likewise empty stubs.
Drop the libdl, threads, and librt entries from every meson.build, and
remove the now-stale 'Libs.private: -lrt -pthread' from libudev.pc.in
since both flags resolve to empty link-time stubs on glibc 2.34+ and
musl.
Verified with readelf -d that libsystemd.so, libudev.so, and systemd no
longer carry DT_NEEDED entries for libdl/libpthread/librt.
dgettext() lives in libc on glibc and in libintl.so.8 on musl with
gettext. Resolve it via dlsym() so neither configuration produces a
hard link-time dependency on libintl: try libintl.so.8 first and fall
back to RTLD_DEFAULT (which finds dgettext in libc on glibc).
The _() macro now expands to a runtime check that returns the
untranslated string if dlopen_libintl() has not run successfully, so
callers don't have to gate every translatable message on a runtime
check. pam_systemd_home — currently the only consumer of _() — calls
dlopen_libintl() best-effort from each PAM entry point.
The meson find_library('intl') dance is replaced with a has_header()
check; the only thing we need at build time is the prototype.
The test only uses 9 symbols (5 from glib, 4 from libdbus) for its
interop checks; dlopen them at runtime so the binary no longer carries a
hard link-time dependency on either library. Headers are still pulled in
through the *_cflags partial dependencies for the type declarations.
While we're at it, drop the compat glue for glib 2.36 which is long obsolete
at this point.
Same reasoning as for cryptsetup tokens. It means we can include
the pam plugins in the main systemd package without the package
manager introducing a dependency on libpam. It also makes things
more consistent and makes writing the upcoming linking test script
a lot simpler.
At the same time, we get rid of the libpam_misc dependency as the
one symbol we were using from it is trivial to reimplement ourselves.
libgpg-error was added in 2017 (commit 76c8741060, Michael Biebl) to
gate HAVE_GCRYPT on its presence because src/resolve referenced
libgpg-error directly at the time. That usage is long gone — no source
file references any gpg-error API today — so the dependency only served
to fail HAVE_GCRYPT detection when gpg-error-dev wasn't installed.
libgcrypt's pkg-config Requires already pulls in the gpg-error headers
(via the transitive #include <gpg-error.h> in <gcrypt.h>), so dropping
the dep doesn't break compilation.
Checking for pefile required that module to be made available for the
Python used to build systemd, even though it's only used at runtime,
potentially via a different Python installation.
Furthermore, Meson's Python module doesn't do the right thing when cross
compiling and looking up a Python for the host system, so this would end
up uselessly checking whether the build Python had the pefile module,
which is not needed. Even if it were made to check the host Python using
find_program, it still relies on being able to run its Python, which in
a cross scenario it probably wouldn't be able to do.
All in all, this check does more harm than good, and prevents building
ukify in valid configurations, so remove it.
For libc syscall wrappers (pidfd_open, fsopen, openat2, etc.) we previously
gated the calls behind build-time HAVE_* checks. Replace these with shim
functions in src/libc/ that fall back to the raw syscall at runtime when the
loaded glibc lacks the symbol. The infrastructure lives in src/libc/libc-shim.h:
DEFINE_SYSCALL_SHIM falls back to a direct syscall, DEFINE_LIBC_SHIM returns
ENOSYS (for posix_spawn-family helpers that have no corresponding syscall), and
DEFINE_LIBC_ERRNO_SHIM sets errno=ENOSYS and returns -1 (for read/write-style
helpers). The weak reference to the libc symbol is bound via __asm__(\"name\")
rename so the bare libc identifier never appears as a C token — this avoids
both #undef boilerplate against override-header redirects and the resulting
-Wredundant-decls warning. Drop the corresponding cc.has_function() loop from
meson.build.
For optional libraries (libcryptsetup, libdw, libarchive), drop the per-symbol
HAVE_* checks. Always declare the prototypes, suppressing the redundant-decl
warnings via DISABLE_WARNING_REDUNDANT_DECLS and NOLINT, and resolve the symbols
after the main dlopen via a new DLSYM_OPTIONAL() helper that only assigns on
success. libcryptsetup's crypt_set_keyring_to_link / crypt_token_set_external_path
and libarchive's *_is_set wrappers use fallback functions as their pointer
initializers (returning -ENOSYS and 0 respectively), so call sites can invoke
the symbol unconditionally and just check for -ENOSYS where the \"not supported\"
distinction matters.
The same shim treatment applies to pidfd_spawn / posix_spawnattr_setcgroup_np
(src/libc/spawn.c) and epoll_pwait2 (src/libc/epoll.c), with corresponding
override headers in src/include/override/spawn.h and
src/include/override/sys/epoll.h. posix_spawn_wrapper() in process-util.c and
epoll_wait_usec() in sd-event.c now detect ENOSYS in the return value instead
of checking the function pointer, falling back to plain posix_spawn() and
epoll_wait() respectively. coredump-config and coredump-submit get a
dlopen_dw_has_dwfl_set_sysroot() helper. The kexec arch gate now uses
defined(__NR_kexec_file_load) directly; pidfd.h uses __has_include_next() to
decide whether to pull in glibc's header.
This lets binaries built against newer glibc / libcryptsetup / libdw /
libarchive headers still load and run on older targets where these symbols are
absent.
Because fdisk_assign_device tries to open block devices with O_EXCL, when it
does it blocks cryptsetup from using partition block devices for the same
disk.
Since we already have a file descriptor for the device, we can just share it
and use fdisk_assign_device_by_fd instead.
This requires at least libfdisk 2.35 (part of util-linux) which was
released in 2020.
Previously, if boot was enabled but ukify wasn't, the check for the pefile
Python module would be omitted. The check in want_ukify was incorrect — with
Meson it's only possible to check for Python dependencies available for the
build Python. When the build Python is not the same as the Python that will be
used to run the installed ukify, this would incorrectly require ukify to be
available to the build Python. The most common, but not the only case where
this would happen would be when cross compiling. If bootloader and tests are
disabled, there's no need for the build Python to have pefile even when
installing ukify. Therefore, check in the more specific case of ukify's tests
being enabled, in which case pefile is required at build time.
First implementation of io.systemd.StorageProvider, exposing all block
devices known to udev (disks, partitions, dm nodes, …) as volumes of
type "blk". Names are picked from stable /dev/mapper and /dev/disk/by-*
symlinks; content-derived identifiers (by-uuid, by-label, …) are
intentionally avoided for security. Volume creation is not supported by
this backend.
Socket-activated via /run/systemd/io.systemd.StorageProvider/block.
Also adds shared storage-util.[ch] (VolumeType / CreateMode helpers)
that subsequent providers reuse.
Until now OpenSSL was linked into every binary and library that needed
cryptography, pulling libcrypto (and, for resolved, libssl) into the
address space of services that never touch them at runtime. This commit
moves all OpenSSL usage behind the same dlopen helper pattern that we
already use for other optional libraries (libpam, libseccomp, libxz, …)
so libcrypto/libssl are only loaded on demand.
The bulk of the work lives in src/shared/crypto-util.{c,h} (libcrypto)
and src/shared/ssl-util.{c,h} (libssl), which replace the previous
src/shared/openssl-util.{c,h}:
- crypto-util.{c,h} declares every libcrypto function we call via
DLSYM_PROTOTYPE() and resolves them inside dlopen_libcrypto().
- ssl-util.{c,h} holds the libssl-specific DLSYM_PROTOTYPEs,
dlopen_libssl(), and the SSL_freep cleanup helper, so translation
units that only need libcrypto do not pull in libssl declarations.
- Callers refer to the symbols through sym_* aliases rather than the
original names.
- Convenience macros that used to be provided by the OpenSSL headers
(OPENSSL_free, BN_num_bytes, the sk_TYPE_* helpers, …) are
reimplemented as sym_* wrappers so no code path needs to fall back
to the linker-resolved symbols.
- All _cleanup_ helpers are redefined in terms of the sym_* variants
(EVP_PKEY_freep, X509_freep, BIO_freep, …) so cleanup attributes
keep working without pulling in libcrypto symbols at link time.
- The public crypto-util.c entry points (openssl_pubkey_from_pem,
openssl_digest_many, openssl_hmac_many, openssl_cipher_many,
kdf_ss_derive, kdf_kb_hmac_derive, rsa_* / ecc_* helpers,
pubkey_fingerprint, digest_and_sign, pkcs7_new, x509_fingerprint,
openssl_extract_public_key, pkey_generate_volume_keys, the load_*
helpers, …) now call dlopen_libcrypto() at entry before touching any
sym_* pointer.
The call sites across the tree have been converted to call
dlopen_libcrypto()/dlopen_libssl() at the appropriate entry point
before their first sym_* use, and to use sym_* variants throughout:
- bootctl, sbsign, measure, pcrlock, pcrextend, tpm2-setup, repart,
cryptsetup, cryptenroll, homectl, homed, homework, keyutil,
sysupdate, creds, import, dissect-image, pe-binary, pkcs11-util,
pkcs7-util, tpm2-util, creds-util. resolved additionally dlopens
libssl for DoT.
The meson build files are updated to depend on libopenssl_cflags (a
new partial dependency that exposes include paths and compile flags
only, not the linker flags) instead of libopenssl for every target that
previously linked against OpenSSL. Nothing links against libcrypto or
libssl directly anymore.
A new src/sbsign/authenticode.c hosts the Authenticode ASN.1 type
definitions that used to live inline in sbsign.c. The OpenSSL
ASN1_SEQUENCE / ASN1_CHOICE / IMPLEMENT_ASN1_FUNCTIONS macros expand to
code that references libcrypto symbols directly, so to keep this
translation unit unlinked from libcrypto we redirect ASN1_item_* to
the sym_* variants via #define and wrap the ASN1_*_it() getters (which
appear as constant function pointers in static initializers) in small
trampoline functions that forward to the sym_* pointers at runtime.
test-dlopen-so gains assertions for dlopen_libcrypto and dlopen_libssl
so the dlopen contract is exercised in CI, and the openssl-specific
test was renamed from test-openssl.c to test-crypto-util.c to match
the new header naming.
Convert the GnuTLS and libmicrohttpd usage in journal-remote to the
dlopen pattern used by other optional shared libraries. A new
src/shared/gnutls-util.{h,c} declares the GnuTLS entry points via
DLSYM_PROTOTYPE and resolves them in dlopen_gnutls(); microhttpd-util
is moved from src/journal-remote to src/shared and gains analogous
DLSYM_PROTOTYPEs plus dlopen_microhttpd(). Callers in journal-gatewayd,
journal-remote-main and microhttpd-util itself call the sym_* wrappers
and invoke dlopen_gnutls()/dlopen_microhttpd() at their entry points.
setup_gnutls_logger() no longer fails when libgnutls is missing at
runtime; it logs a notice and returns 0 so journal-gatewayd starts up
without TLS dependencies installed.
The meson files gain libgnutls_cflags and libmicrohttpd_cflags partial
dependencies that expose include paths and compile flags only. Every
systemd-journal-{gatewayd,remote,upload} target switches to the cflags
variant, dropping the direct libgnutls/libmicrohttpd link. The
gatewayd->remote object-sharing dance for microhttpd-util.o goes away
since the code now lives in libshared.
test-dlopen-so gains assertions for dlopen_gnutls and dlopen_microhttpd.
The six .bpf.c files and their shared .bpf.h headers now live directly
under src/bpf/, rather than scattered across src/core/bpf/<prog>/,
src/network/bpf/<prog>/ and src/nsresourced/bpf/<prog>/.
All BPF compilation logic — the BPF_FRAMEWORK determination (clang/gcc/
bpftool/llvm-strip lookups), flag and command construction, vmlinux.h
handling, the bpf_programs list and the loop that builds the unstripped
object, the stripped object and the skeleton header for each program —
moves into src/bpf/meson.build. The top-level meson.build only keeps
option handling and the libbpf dependency. subdir('src/bpf') is pulled
up before config.h generation so that BPF_FRAMEWORK, HAVE_VMLINUX_H and
ENABLE_SYSCTL_BPF land in conf in time.
Skeleton wrapper headers (<prog>-skel.h) are now emitted by a
configure_file() template (src/bpf/bpf-skel-wrapper.h.in) at meson
setup, replacing the previously checked-in shims of the same name.
Consumer #include paths are flattened:
"bpf/<prog>/<prog>-skel.h" becomes "<prog>-skel.h",
"bpf/<prog>/<prog>-api.bpf.h" becomes "<prog>-api.bpf.h". src/bpf is
added to includes so the shared BPF headers resolve.
sysctl-write-event.h, now shared between userspace (networkd-sysctl.c)
and BPF (sysctl-monitor.bpf.c) from a single location, gains guarded
includes so pid_t and uint64_t resolve on both sides: vmlinux.h in the
BPF case (selected via __bpf__), stdint.h + sys/types.h otherwise.
Convert fdisk-util to the dlopen pattern used by other optional shared
libraries in libshared. Declare the libfdisk API entry points with
DLSYM_PROTOTYPE, resolve them in a dlopen_fdisk() helper, and call the
sym_* wrappers from the homework, sysupdate and repart binaries that
use them.
With this in place fdisk-util can live in libshared itself, linked only
against libfdisk's headers (via libfdisk_cflags). The libshared_fdisk
convenience library and the libfdisk link dependency on systemd-homework,
systemd-sysupdate, systemd-repart and systemd-repart.standalone go away.
Also add a dlopen_fdisk() check to test-dlopen-so.
Convert curl-util to the dlopen pattern used by other optional shared
libraries in libshared (libarchive, pcre2, idn, ...). Declare the curl
API entry points with DLSYM_PROTOTYPE, resolve them in a dlopen_curl()
helper, and call the sym_* wrappers from callers. curl_glue_new() now
loads the library on first use, so consumers going through CurlGlue
pick this up automatically; journal-upload and report-upload call
dlopen_curl() directly since they use curl without the glue layer.
With this in place curl-util can live in libshared itself, linked only
against libcurl's headers (via libcurl_cflags). The libcurlutil_static
convenience library and the libcurl link dependency on systemd-imdsd,
systemd-pull, systemd-journal-upload and systemd-report go away.
Also move the easy_setopt() helper macro next to the DLSYM declarations
so all consumers use a single sym-prefixed definition, and add a
dlopen_curl() check to test-dlopen-so.
Otherwise you run into errors such as:
"""
../meson.build:2899:28: ERROR: File src/test/test-loop-util.c does not exist.
"""
when deleting a file in git without staging the deletion.
Move the push-based streaming compression API from import-compress.c
into compress.c and delete import-compress.c/h. This consolidates all
compression code in one place and makes all compression libraries
(liblzma, liblz4, libzstd, libz, libbz2) runtime-loaded via dlopen
instead of directly linked.
Introduce opaque Compressor/Decompressor types backed by a heap-
allocated struct defined only in compress.c, keeping all third-party
library headers out of compress.h.
Rewrite the per-codec fd-to-fd stream functions as thin wrappers around
the push API via generic compress_stream()/decompress_stream() taking a
Compression type parameter. Integrate LZ4 into this framework using the
LZ4 Frame API, eliminating all LZ4 special-casing.
Extend the Compression enum with COMPRESSION_GZIP and COMPRESSION_BZIP2
and add the corresponding blob, startswith, and stream functions for
both.
Rename the ImportCompress types and functions: ImportCompressType becomes
the existing Compression enum, ImportCompress becomes Compressor (with
Decompressor typedef), and all import_compress_*/import_uncompress_*
become compressor_*/decompressor_*. Rename dlopen_lzma() to dlopen_xz()
for consistency. Make compression_to_string() return lowercase by
default.
Add INT_MAX/UINT_MAX overflow checks for LZ4, zlib, and bzip2 blob
functions where the codec API uses narrower integer types than our
uint64_t parameters.
Migrate test-compress.c and test-compress-benchmark.c to the TEST()
macro framework, new assertion macros, and codec-generic loops instead
of per-codec duplication.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
In some builds (package builds, so with optimization and lto, but I
haven't been able to pin down the exact combination on options that
matters), we end up with items in the verbs array reordered. The order
matters (because of groups, but also because we have some specific order
for display), so this reordering is something that we don't want.
From what I was able to read, the compiler + linker generally keep the
order within a single translation unit, but this is more of a convention
and implementation choice than a guarantee. Add this attribute [1]. It
seems to have the desired effect in CI.
[1] https://gcc.gnu.org/onlinedocs/gcc-15.2.0/gcc/Common-Function-Attributes.html#index-no_005freorder-function-attribute
Add the needed assert changes to make the code clean
for the new check-pointer-deref script.
Note that for `_cmp()` functions its a bit of a question
if we want `assert()` or `POINTER_MAY_BE_NULL()` here. The
former is more correct but it adds a (small) runtime hit. If
we use POINTER_MAY_BE_NULL() it is compiled out but strictly
speaking it is wrong (so at least we would need a comment).
For some of the directories it makes more sense to keep them
excluded from the coccinelle check. Specifically:
- libc: compatibility, no asserts or systemd headers yet
- test: uses NUL internally to test crashes etc
Daan De Meyer
Luca Boccassi
Yu Watanabe
Valentin David
Alyssa Ross
Lennart Poettering
Lennart Poettering
Zbigniew Jędrzejewski-Szmek
Michael Vogt