Covers the SCM_RIGHTS fd-passing path end-to-end against a real QEMU: open an
eventfd, hand it off via QMP_CLIENT_ARGS_FD() on the very first qmp_client_invoke()
against a fresh client, and verify QEMU's add-fd reply carries the expected
fdset-id. Complements the mock-based unit test with an authoritative check that
QEMU actually consumes the fd from its FIFO receive queue when processing the
command — the AF_UNIX kernel behaviour around non-scm skb absorption into
following scm-bearing recvs is real-traffic-shaped rather than mock-shaped.
QEMU's QMP greeting is an unsolicited, informational, server-initiated advertisement
— it doesn't gate commands. The server accepts (and pipelines) commands the instant
the socket is open. We were using it as a trigger to build qmp_capabilities and
blocking callers via qmp_client_ensure_running() until the reply came back.
Build qmp_capabilities inside qmp_client_connect_fd() instead and let the JsonStream
output queue preserve FIFO ordering between it and any user command a subsequent
invoke() enqueues. That satisfies QEMU's only ordering requirement (cap must precede
other commands) without any blocking in the send path.
Fallout:
- Collapse QmpClientState to {RUNNING, DISCONNECTED}. The three HANDSHAKE_* states
and the QMP_CLIENT_STATE_IS_HANDSHAKE() macro go away.
- Drop qmp_client_dispatch_handshake(); fold greeting-drop into dispatch_reply as
a one-line shape check.
- Drop qmp_client_ensure_running() and its qmp_client_send() call site. send()
now only refuses when state == DISCONNECTED.
- The qmp_capabilities reply lands on an ordinary slot whose callback logs a
protocol-level error and force-disconnects if cap negotiation failed, matching
the old EPROTO behaviour at the same observable boundary.
- qmp_client_phase() no longer special-cases the old handshake states; it maps
directly to READING / AWAITING_REPLY based on whether slots are in flight.
Test updates:
- qmp_client_first_invoke_with_fd → qmp_client_invoke_with_fd. The scenario it
was pinned to (push_fd+invoke staging order) has been structurally impossible
since the QmpClientArgs rework in 8ad4adcb6f; eager-cap removes it a second
way. The test now covers end-to-end fd-passing on the first invoke, accepting
either recv carrying the single SCM_RIGHTS fd (AF_UNIX absorbs non-scm skbs
forward into the next scm-bearing skb's recv, so the fd may surface with cap
or add-fd depending on kernel scheduling — QEMU's FIFO fd queue handles either).
- qmp_client_invoke_failure_closes_fds restructured around the new invariant:
invoke no longer blocks and no longer returns ENOTCONN for a dead peer, so
the fd-leak assertion moves to "still open while the JsonStream queue owns it,
closed on client teardown" and the nested block is flattened into an explicit
qmp_client_unref().
QmpSlot now stores a back-reference to its QmpClient (mirroring sd_bus_slot)
and is exposed as a public refcounted type via qmp_slot_ref/qmp_slot_unref.
qmp_client_invoke() gains an optional QmpSlot **ret_slot out-parameter
matching sd_bus_call_async(): passing non-NULL hands back a reference whose
unref cancels the pending call (the callback is deregistered; a late reply
is logged and discarded as unknown-id).
Internally slots come in two flavours, following sd_bus's model: floating
(owned by the client's pending set, used when ret_slot is NULL) and
non-floating (ref held by the caller, slot holds a ref on the client).
qmp_slot_disconnect() centralizes the teardown so the reply-dispatched,
explicit-cancel, and client-teardown paths all converge on the same
idempotent cleanup.
qmp_client_call()'s sync slot is now non-floating and observes completion
by watching slot->client go NULL instead of set_contains() on an id.
The mock servers previously framed messages by hand: loop_write()+"\r\n" on
the way out, a single read(fd, buf, 4095)+sd_json_parse() on the way in,
and a custom recvmsg()+CMSG_FOREACH() for the fd-passing test. That only
works when each write-on-the-wire happens to be delivered in its own
recv(); the moment a test wants to issue back-to-back commands without
waiting for replies, the kernel coalesces them and sd_json_parse() chokes
on two concatenated objects.
Route the mock servers through the same JsonStream transport the client
uses: a tiny mock_qmp_init/recv/send/send_literal layer over json_stream
takes care of the CRLF delimiter, the output queue, and SCM_RIGHTS. The
recv helper loops parse→read→wait so coalesced inbound bytes get fed out
one complete message at a time.
Drop the bespoke mock_qmp_recv_command() and replace it with
json_stream_set_allow_fd_passing_input() +
json_stream_{get_n,take,close}_input_fds() in the fd-first test. EOF
signalling moves from an explicit safe_close() to the
_cleanup_(json_stream_done) on the JsonStream.
Until now OpenSSL was linked into every binary and library that needed
cryptography, pulling libcrypto (and, for resolved, libssl) into the
address space of services that never touch them at runtime. This commit
moves all OpenSSL usage behind the same dlopen helper pattern that we
already use for other optional libraries (libpam, libseccomp, libxz, …)
so libcrypto/libssl are only loaded on demand.
The bulk of the work lives in src/shared/crypto-util.{c,h} (libcrypto)
and src/shared/ssl-util.{c,h} (libssl), which replace the previous
src/shared/openssl-util.{c,h}:
- crypto-util.{c,h} declares every libcrypto function we call via
DLSYM_PROTOTYPE() and resolves them inside dlopen_libcrypto().
- ssl-util.{c,h} holds the libssl-specific DLSYM_PROTOTYPEs,
dlopen_libssl(), and the SSL_freep cleanup helper, so translation
units that only need libcrypto do not pull in libssl declarations.
- Callers refer to the symbols through sym_* aliases rather than the
original names.
- Convenience macros that used to be provided by the OpenSSL headers
(OPENSSL_free, BN_num_bytes, the sk_TYPE_* helpers, …) are
reimplemented as sym_* wrappers so no code path needs to fall back
to the linker-resolved symbols.
- All _cleanup_ helpers are redefined in terms of the sym_* variants
(EVP_PKEY_freep, X509_freep, BIO_freep, …) so cleanup attributes
keep working without pulling in libcrypto symbols at link time.
- The public crypto-util.c entry points (openssl_pubkey_from_pem,
openssl_digest_many, openssl_hmac_many, openssl_cipher_many,
kdf_ss_derive, kdf_kb_hmac_derive, rsa_* / ecc_* helpers,
pubkey_fingerprint, digest_and_sign, pkcs7_new, x509_fingerprint,
openssl_extract_public_key, pkey_generate_volume_keys, the load_*
helpers, …) now call dlopen_libcrypto() at entry before touching any
sym_* pointer.
The call sites across the tree have been converted to call
dlopen_libcrypto()/dlopen_libssl() at the appropriate entry point
before their first sym_* use, and to use sym_* variants throughout:
- bootctl, sbsign, measure, pcrlock, pcrextend, tpm2-setup, repart,
cryptsetup, cryptenroll, homectl, homed, homework, keyutil,
sysupdate, creds, import, dissect-image, pe-binary, pkcs11-util,
pkcs7-util, tpm2-util, creds-util. resolved additionally dlopens
libssl for DoT.
The meson build files are updated to depend on libopenssl_cflags (a
new partial dependency that exposes include paths and compile flags
only, not the linker flags) instead of libopenssl for every target that
previously linked against OpenSSL. Nothing links against libcrypto or
libssl directly anymore.
A new src/sbsign/authenticode.c hosts the Authenticode ASN.1 type
definitions that used to live inline in sbsign.c. The OpenSSL
ASN1_SEQUENCE / ASN1_CHOICE / IMPLEMENT_ASN1_FUNCTIONS macros expand to
code that references libcrypto symbols directly, so to keep this
translation unit unlinked from libcrypto we redirect ASN1_item_* to
the sym_* variants via #define and wrap the ASN1_*_it() getters (which
appear as constant function pointers in static initializers) in small
trampoline functions that forward to the sym_* pointers at runtime.
test-dlopen-so gains assertions for dlopen_libcrypto and dlopen_libssl
so the dlopen contract is exercised in CI, and the openssl-specific
test was renamed from test-openssl.c to test-crypto-util.c to match
the new header naming.
Add a synchronous counterpart to qmp_client_invoke() that pumps the
client's own process()/wait() loop until the reply for the issued
command id arrives, mirroring sd_varlink_call()'s contract: *ret_result
and *ret_error_desc are borrowed pointers into c->current, valid until
the next qmp_client_call(), and a QMP error surfaces as -EIO when the
caller doesn't ask for the description.
Factor the command-build + slot-insert + enqueue sequence shared with
qmp_client_invoke() into qmp_client_send(). A NULL callback marks the
slot as synchronous: dispatch_reply still matches on id (so unknown
ids continue to be logged and discarded, preserving async-only
robustness), but skips the TAKE_PTR + callback invocation and leaves
c->current pinned for qmp_client_call() to read out.
Cover the three paths in test-qmp-client: successful reply, QMP error
with ret_error_desc, and QMP error returned as -EIO.
Convert the GnuTLS and libmicrohttpd usage in journal-remote to the
dlopen pattern used by other optional shared libraries. A new
src/shared/gnutls-util.{h,c} declares the GnuTLS entry points via
DLSYM_PROTOTYPE and resolves them in dlopen_gnutls(); microhttpd-util
is moved from src/journal-remote to src/shared and gains analogous
DLSYM_PROTOTYPEs plus dlopen_microhttpd(). Callers in journal-gatewayd,
journal-remote-main and microhttpd-util itself call the sym_* wrappers
and invoke dlopen_gnutls()/dlopen_microhttpd() at their entry points.
setup_gnutls_logger() no longer fails when libgnutls is missing at
runtime; it logs a notice and returns 0 so journal-gatewayd starts up
without TLS dependencies installed.
The meson files gain libgnutls_cflags and libmicrohttpd_cflags partial
dependencies that expose include paths and compile flags only. Every
systemd-journal-{gatewayd,remote,upload} target switches to the cflags
variant, dropping the direct libgnutls/libmicrohttpd link. The
gatewayd->remote object-sharing dance for microhttpd-util.o goes away
since the code now lives in libshared.
test-dlopen-so gains assertions for dlopen_gnutls and dlopen_microhttpd.
Every dlopen_xxx() helper now takes an int log_level argument. It is
passed through to dlopen_many_sym_or_warn() (which in turn propagates it
to dlopen_verbose() for the library-not-installed case), and is used by
the fallback stub when support for the library is not compiled in to
emit a "<lib> support is not compiled in." message at the caller's level.
Callers pass LOG_DEBUG when gracefully degrading, or a higher level when
the failure should surface, and no longer need to log redundantly at the
call site.
As part of this, dlopen_bpf_full() (which already took a log_level) is
merged into dlopen_bpf() rather than keeping both.
The static inline fallbacks used to live in the headers, which required
pulling log.h in from every header that declared a dlopen_xxx(). Move
them into the .c files instead: the declaration is always outside the
#if HAVE_XXX block, the impl sits outside the outer #if HAVE_XXX wrap
with its own internal #if HAVE_XXX/#else/#endif, and apparmor-util.c,
idn-util.c, libmount-util.c and pam-util.c are now always compiled so
they can host their stubs.
Convert fdisk-util to the dlopen pattern used by other optional shared
libraries in libshared. Declare the libfdisk API entry points with
DLSYM_PROTOTYPE, resolve them in a dlopen_fdisk() helper, and call the
sym_* wrappers from the homework, sysupdate and repart binaries that
use them.
With this in place fdisk-util can live in libshared itself, linked only
against libfdisk's headers (via libfdisk_cflags). The libshared_fdisk
convenience library and the libfdisk link dependency on systemd-homework,
systemd-sysupdate, systemd-repart and systemd-repart.standalone go away.
Also add a dlopen_fdisk() check to test-dlopen-so.
Split the single directory fd that chaseat() used to take into two separate
fds: a root_fd that sets the chroot boundary (symlinks may not escape it,
absolute symlinks resolve relative to it), and a dir_fd that path resolution
starts from. This makes the chroot semantics of chaseat() explicit at every
call site instead of encoding them in the CHASE_AT_RESOLVE_IN_ROOT flag,
which is removed. It also decouples the starting directory from the root
boundary, so callers can descend from any inode inside the tree without
having to reopen the root separately.
XAT_FDROOT passed as root_fd means "no containment" (host root); as dir_fd
it means "start at root_fd". For a smoother transition, AT_FDCWD is also
accepted as root_fd and treated as XAT_FDROOT. When root_fd points to a
directory that is actually the host root, it is normalized to XAT_FDROOT
up front so the existing shortcut path can kick in.
Absolute paths returned by chaseat() are now relative to root_fd, and
relative paths are relative to dir_fd. The result is absolute only when
there is no chroot boundary (root_fd is XAT_FDROOT), or when an absolute
symlink made resolution jump out of the dir_fd subtree; otherwise callers
get a relative path they can feed straight back into an openat()-style
call against dir_fd. Specifically, when dir_fd == root_fd and we're not
operating on the host's root directory, we return a relative path even if
we received an absolute path or resolved an absolute symlink to allow
passing the path directly to openat() style functions. We do this to not
have to go modify every caller of chaseat() to make sure they deal properly
with any absolute paths they might receive. Only when root_fd != dir_fd do
we have to return an absolute path to indicate that the path is relative to
root_fd and not dir_fd.
The shortcut that skips the per-component walk is reworked around a new
chase_xopenat() helper that funnels CHASE_NOFOLLOW, CHASE_MUST_BE_* and
CHASE_TRIGGER_AUTOFS through xopenat_full()'s O_NOFOLLOW, O_DIRECTORY,
XO_REGULAR, XO_SOCKET and XO_TRIGGER_AUTOMOUNT flags. As a result these
flags no longer force us off the shortcut and can be dropped from
CHASE_NO_SHORTCUT_MASK, and the old openat_opath_with_automount() helper
goes away. A CHASE_MUST_BE_ANY alias is introduced for shortcut callers
(stat/access paths) that don't go through xopenat_full() and still need
to bail on those flags locally.
All *_and_* helpers built on top of chaseat() (chase_and_openat,
chase_and_opendirat, chase_and_statat, chase_and_accessat,
chase_and_fopenat_unlocked, chase_and_unlinkat, chase_and_open_parent_at)
gain the same root_fd parameter, and every call site in the tree is
ported to the new signature.
chase_and_open() is also fixed to correctly handle CHASE_EXTRACT_FILENAME
without CHASE_PARENT.
XO_SOCKET asks for the opened inode to be verified as a socket, analogous to
the existing XO_REGULAR. A new fd_verify_socket() helper is added to
stat-util to perform the check on an opened fd.
XO_TRIGGER_AUTOMOUNT asks O_PATH opens to trigger automounts by going via
open_tree() without OPEN_TREE_CLONE, falling back to openat() if the kernel
or sandbox rejects open_tree().
Convert curl-util to the dlopen pattern used by other optional shared
libraries in libshared (libarchive, pcre2, idn, ...). Declare the curl
API entry points with DLSYM_PROTOTYPE, resolve them in a dlopen_curl()
helper, and call the sym_* wrappers from callers. curl_glue_new() now
loads the library on first use, so consumers going through CurlGlue
pick this up automatically; journal-upload and report-upload call
dlopen_curl() directly since they use curl without the glue layer.
With this in place curl-util can live in libshared itself, linked only
against libcurl's headers (via libcurl_cflags). The libcurlutil_static
convenience library and the libcurl link dependency on systemd-imdsd,
systemd-pull, systemd-journal-upload and systemd-report go away.
Also move the easy_setopt() helper macro next to the DLSYM declarations
so all consumers use a single sym-prefixed definition, and add a
dlopen_curl() check to test-dlopen-so.
The naming is consistent with strv_extend().
This also
- introduces tiny iovw_extend_iov() wrapper,
- refuse when the source and target points to the same object,
- check the final count before extending in iovw_extend_iovw().
This is similar to iovw_to_cstring(), but allows embedded NUL, as this
just concat multiple iovec, the result may not be a string.
Now, iovw_to_cstring() internally uses iovw_concat().
This simply mirrors the same overflow check we already have in
strrepa(), in case someone passed us a sufficiently long string.
strrep() is currently used only in tests, so this is just hardening.
This simply mirrors the same overflow check we already have in
strrepa(), in case someone passed us a sufficiently long string.
strrep() is currently used only in tests, so this is just hardening.
All non-test users iovec_wrapper define the struct as a field in a
bigger structure, so we never free it individually. Let's simplify the
code and assume it is never null.
The parsing "mode" is specified as an exclusive mode, i.e. a combination
of "+" and "-" is not supported. In principle this could be supported,
but we don't use that in our code and are unlikely ever to do so.
This will make it easier to add new modes of operation later.
But I'm happy with how this came out — I think the mode setting
is nicer to read then the old bool.
Add a test that launches QEMU with -machine none (no bootable image
needed) and exercises the QMP client library against the real QMP
implementation:
- test_qmp_client_qemu_handshake_and_schema: sends query-qmp-schema
(~200KB response that exercises the buffered multi-read() path)
via qmp_client_invoke(), then cleanly shuts down QEMU via quit.
The QMP handshake completes transparently inside invoke().
- test_qmp_client_qemu_query_status: validates query-status response
parsing, stop/cont command sequencing with id correlation, and state
verification between commands
The test is automatically skipped when QEMU is not installed.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Test the QMP client library using a mock QMP server over a socketpair:
- test_qmp_client_basic: Verifies full handshake, query-status with
response parsing, stop/cont commands, and asynchronous STOP event
delivery via the sd-event I/O callback
- test_qmp_client_eof: Verifies that the client properly detects
server disconnection (EOF) and returns a disconnect error
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Convert the class field (Register input, List output) from plain string
to MachineClass enum type, and the whom field (Kill input) to KillWhom
enum type.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Convert CPUSchedulingPolicy, IOSchedulingClass, NUMAPolicy and MountFlags
fields from plain strings to proper varlink enum types in the io.systemd.Unit
interface. Update the corresponding serialization code to use
json_underscorify() for correct enum value formatting.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Convert 8 string fields in the io.systemd.Manager varlink interface to
proper enum types:
- LogTarget: new enum (console, console_prefixed, kmsg, journal, ...)
- DefaultStandardOutput/Error: reuse ExecOutputType from common
- DefaultMemory/CPU/IOPressureWatch: reuse CGroupPressureWatch from common
- DefaultOOMPolicy: new enum (continue, stop, kill)
- CtrlAltDelBurstAction: reuse EmergencyAction from common
Output serialization updated to use JSON_BUILD_PAIR_ENUM for automatic
underscorification of dash-containing values.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Move ExecOutputType, CGroupPressureWatch, EmergencyAction and
ManagedOOMMode enum type definitions from varlink-io.systemd.Unit to
varlink-idl-common, as these types are shared across multiple varlink
interfaces.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace the fork+exec of /usr/bin/kexec in load_kexec_kernel() with a
direct kexec_file_load() syscall, removing the runtime dependency on
kexec-tools for systemctl kexec.
The kexec_file_load() syscall (available since Linux 3.17) accepts
kernel and initrd file descriptors directly, letting the kernel handle
image parsing, segment setup, and purgatory internally. This is much
simpler than the older kexec_load() syscall which requires complex
userspace setup of memory segments and boot protocol structures — that
complexity is the raison d'être of kexec-tools.
The implementation follows the established libc wrapper pattern: a
missing_kexec_file_load() fallback in src/libc/kexec.c calls the
syscall directly when glibc doesn't provide a wrapper (which is
currently always the case). The syscall is not available on all
architectures — alpha, i386, ia64, m68k, mips, sh, and sparc lack
__NR_kexec_file_load — so the wrapper and caller are guarded with
HAVE_KEXEC_FILE_LOAD_SYSCALL to compile cleanly everywhere.
When kexec_file_load() rejects the kernel image with ENOEXEC (e.g. the
image is compressed or wrapped in a PE container that the kernel's kexec
handler doesn't understand natively), we attempt to unwrap/decompress
and retry. This is effectively the same decompression and extraction
logic that already lives in src/ukify/ukify.py (maybe_decompress() and
get_zboot_kernel()), now implemented in C so that systemctl can handle
it natively without shelling out to external tools:
- Compressed kernels (Image.gz, Image.zst, Image.xz, Image.lz4): the
format is detected by magic bytes (per RFC 1952, RFC 8878,
tukaani.org xz spec, and lz4 frame format spec) and decompressed to
a memfd using the existing decompress_stream_*() infrastructure plus
the new gzip support from the previous commit. This is primarily
needed on arm64 where kexec_file_load() only accepts raw Image files.
On x86_64, bzImage is already the native format and works directly.
- EFI ZBOOT PE images (vmlinuz.efi): detected by "MZ" + "zimg" magic
at the start of the file. The compressed payload offset, size, and
compression type are read from the ZBOOT header defined in
linux/drivers/firmware/efi/libstub/zboot-header.S.
- Unified Kernel Images (UKI): detected as PE files with a .linux
section via the existing pe_is_uki() infrastructure. The .linux
section (kernel) and optionally .initrd section are extracted to
memfds. When a UKI provides an embedded initrd and the boot entry
doesn't specify one separately, the embedded initrd is used.
The try-first-then-decompress approach means we never decompress
unnecessarily: on x86_64 the first kexec_file_load() call succeeds
immediately with the raw bzImage, and on architectures where the
kernel's kexec handler natively understands PE (like LoongArch with
kexec_efi_ops), ZBOOT/UKI images work without decompression too.
If kexec_file_load() is unavailable (architectures without the syscall)
or all attempts fail, we fall back to forking+execing the kexec binary.
This preserves compatibility on architectures like i386 and mips where
only the older kexec_load() syscall exists and kexec-tools is needed to
handle the complex userspace setup.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
One more round, this time with the help of the claudebot, especially for
spelunking in git blame to find the original commit and writing commit
messages from the list of warnings exported from coverity
Co-developed-by: Claude
[claude@anthropic.com](mailto:claude@anthropic.com)
Daan De Meyer
Zbigniew Jędrzejewski-Szmek
Christian Brauner
Lennart Poettering
Yu Watanabe
Frantisek Sumsal
Ivan Kruglov
Christian Brauner
Luca Boccassi