Let's pick up all kinds of sidecars and show them, not just addons
This also fixes some issues regarding "root" directory handling.
In one context we'd resolve a directory claiming it was a "root", but it
wasn't.
Implements: https://github.com/uapi-group/specifications/pull/212
Extend io.systemd.CGroup.CpuUsage from a single per-unit nanosecond
counter to three rows distinguished by a "type" field of "total",
"user", or "system". The values come from cpu.stat's usage_usec,
user_usec and system_usec keys, read in a single keyed-attribute
fetch and cached on each CGroupInfo so each scrape only opens
cpu.stat once per cgroup.
VM-only test that exercises both shipped providers through storagectl:
verifies the well-known sockets exist, lists providers/volumes/
templates, creates and acquires volumes from each template
(sparse-file, allocated-file, directory, subvolume), attaches a loop
device to cover the block provider, and exercises the mount.storage
helper.
This commit adds a simple version of io.systemd.Unit.StartTransient
for varlink. It is similar to the dbus version, but there is a key
difference:
1. Instead of building the unit from key/value properties it
takes a structured json object "UnitContext" with a "Service" field
inside.
It is also only implementing a minimal set of what can be done with a
service.
2. No aux units (for now)
3. When called with --more the varlink socket can notify about
state changes depending on the notify{Job,Unit}Changes parameter
This aligns to the json objects/format from
https://github.com/systemd/systemd/pull/39391
and to show how the format can be shared it adds a new
(minimal) `ServiceContext` that is now part of
`io.systemd.Unit.List()`.
The sanitizer job was FUBAR on Fedora Rawhide (and RHEL 10 to some
extent) due to several changes:
- latest LLVM (v22) introduced a change that occasionally generates a
false-positive warning when running with sanitizers
- several tools had to be "ASan-wrapped" because:
- util-linux started linking against libsystemd which propagated to
other tools depending on its shared libraries (like libmount)
- libssh started depending on libfido2, which depends on libudev; this
then translated to an interesting depedency chain where tpm2 utils got a
dependency on libudev through libcurl -> libssh -> libfido2
- polkit added MemoryMax= to its service file, which is incompatible
with ASan-runs (at least with the current limits)
See the commits for more detailed descriptions.
Also, one note: the santizer job is currently still FUBAR on Fedora
Rawhide (or, more specifically, the TEST-50-DISSECT and TEST-58-REPART),
because mkfs.erofs also gained a dependency on libudev (through libcurl,
see above), but the wrapping currently doesn't work as it also depends
on libqpl which is linked with libtsan (which is incompatible with other
sanitizers). This is currently tracked in
https://bugzilla.redhat.com/show_bug.cgi?id=2461146
LLVM 22 introduced an additional check [0] for ptrace() syscall when
invoking sanitizers [0] which currently produces a false-positive
warning when running some of our units under sanitizers:
[ 47.524680] systemd-timedated[740]: ==740==WARNING: ptrace appears to be blocked (is seccomp enabled?). LeakSanitizer may hang.
[ 47.524680] systemd-timedated[740]: ==740==Child exited with signal 15.
...
[ 1555.734223] systemd-oomd[93]: ==93==WARNING: ptrace appears to be blocked (is seccomp enabled?). LeakSanitizer may hang.
[ 1555.734223] systemd-oomd[93]: ==93==Child exited with signal 15.
...
It is a false positive because we disable the seccomp filters
system-wide for our units in the sanitizer jobs.
Now, from what I've seen so far this happens only in
Type=notify(-reload) units that also utilize bus_event_loop_with_idle().
This, combined with the fact that the ptrace()-check child process from
[0] checks only if the child process was killed by _any_ signal, means
that if the systemd unit exits on its own after becoming idle and then
something sends it SIGTERM (either via explicit `systemctl stop` or
during system shutdown), this SIGTERM might hit the ptrace()-check child
process from the sanitizer handler (as we also send the signal to all
processes in the target cgroup), which the parent process then
mistakenly evaluates as a blocked ptrace() syscall, even though the
check process wasn't killed by SIGSYS.
I filed this as [1] to the LLVM project, but let's also temporarily
ignore the warning in the sanitizer report processing, as it currently
causes annoying test fails.
[0] https://github.com/llvm/llvm-project/commit/a708b4bf21d7c2298224cdacf7d424abc3c8fed4
[1] https://github.com/llvm/llvm-project/issues/193714
The verity signature partition content is a bare JSON object. Repart
pads it with zeros to fill the GPT partition. But when splitting out
the content as an individual file, the padding remains, so it's not
a valid text file.
jq started rejecting files with NUL bytes to fix a security issue:
https://github.com/jqlang/jq/commit/6374ae0bcdfe33a18eb0ae6db28493b1f34a0a5b
Trim the output when writing these files out.
In #39675 the reported fail was as follows:
5580s [ 247.559994] TEST-13-NSPAWN.sh[1858]: Exported 93%.
5580s [ 247.659002] TEST-13-NSPAWN.sh[1858]: Exported 95%.
5580s [ 247.785893] TEST-13-NSPAWN.sh[1858]: Operation completed successfully.
5580s [ 247.923727] TEST-13-NSPAWN.sh[1858]: Exiting.
5580s [ 258.300406] TEST-13-NSPAWN.sh[1074]: + machinectl import-raw /var/tmp/container-export.raw container-raw-reimport
5580s [ 258.323328] TEST-13-NSPAWN.sh[1884]: The 'machinectl import-raw' command has been replaced by 'importctl -m import-raw'. Redirecting invocation.
5580s [ 258.659982] TEST-13-NSPAWN.sh[1884]: Failed to transfer image: Remote peer disconnected
5580s [ 258.734218] TEST-13-NSPAWN.sh[1074]: + at_exit
Turns out that the real reason behind this fail is that the machine was
under heavy load due to a busy-loop from the stub init. The cause of
this is a bug in bash, where running commands that fork (i.e. not
built-ins) can cause a permanent busy-loop due to a desync in trap
handling if you send the signals to the bash process _just right_:
[ 90.855318] TEST-13-NSPAWN.sh[1074]: + machinectl poweroff long-running long-running long-running
[ 90.855318] TEST-13-NSPAWN.sh[1074]: + machinectl reboot long-running long-running long-running
[ 90.928980] systemd-nspawn[1679]: ++ touch /poweroff
[ 90.928980] systemd-nspawn[1679]: +++ touch /reboot
[ 90.928980] systemd-nspawn[1679]: + :
[ 90.928980] systemd-nspawn[1679]: + :
[ 90.928980] systemd-nspawn[1679]: + wait
[ 90.928980] systemd-nspawn[1679]: + :
[ 90.928980] systemd-nspawn[1679]: + :
[ 90.928980] systemd-nspawn[1679]: + wait
[ 90.928980] systemd-nspawn[1679]: + :
[ 90.928980] systemd-nspawn[1679]: + :
[ 90.928980] systemd-nspawn[1679]: + wait
...
$ journalctl --file TEST-13-NSPAWN-1.journal -o short-monotonic --no-hostname --grep "^\+ wait$" | wc -l
349734
So the stub-init was hammering the machine in a tight endless loop,
which then caused systemd-importd to timeout when talking to D-Bus:
[ 258.300096] TEST-13-NSPAWN.sh[1074]: + machinectl import-raw /var/tmp/container-export.raw container-raw-reimport
...
[ 258.415319] systemd-importd[1859]: Unable to request name, failing connection: Method call timed out
[ 258.483662] systemd-importd[1859]: Bus n/a: changing state RUNNING → CLOSING
[ 258.605442] systemd-importd[1859]: Bus n/a: changing state CLOSING → CLOSED
[ 258.659958] TEST-13-NSPAWN.sh[1884]: Failed to transfer image: Remote peer disconnected
Given this is not our issue, let's work around it by using just
built-ins from the trap handlers, which are not susceptible to this bug.
Resolves: #39675
`READY=1` is already correctly emitted when acquiring an update, but was
forgotten to be emitted when subsequently installing that update.
That meant that the state tracking code in `sysupdated` and hence
`updatectl` could not properly report the progress of the install
operation, and hence printed “Already up-to-date” after a successful
update installation, rather than “Done”.
Add a test to catch this in future.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41502
Newer tar started using openat2() via open_subdir() to address
CVE-2025-45582 [0]. Now, gnulib, that tar uses, provides the openat2()
syscall in two ways [1]:
1) If glibc doesn't provide openat2(), it provides its own version in
openat2.c, that tries to call openat2() syscall first, and if it
returns ENOSYS, it emulates the function in userspace.
2) If glibc provides openat2(), it uses that directly, without providing
any fallback on ENOSYS.
Quite recently our test suite started calling nspawn with
--suppress-sync=yes. This means that we call seccomp_suppress_sync(),
which eventually calls block_open_flag(), that blocks the openat2()
syscall completely and refuses it with ENOSYS as this syscall can't be
sensibly filtered (see the openat2()-relevant comments in
block_open_flag() and seccomp_restrict_sxid()). And when glibc provides
openat2(), there's no fallback, so the ENOSYS bubbles up to the user as:
TEST-25-IMPORT.sh[163]: + tar xzf /var/tmp/scratch.tar.gz
TEST-25-IMPORT.sh[163]: tar: ./adirectory/athirdfile: Cannot open: Function not implemented
TEST-25-IMPORT.sh[163]: tar: Exiting with failure status due to previous errors
Let's mitigate this by re-enabling sync for TEST-25-IMPORT, at least for
now.
[0] https://cgit.git.savannah.gnu.org/cgit/tar.git/commit/?id=75b03fdff48916bd0654677ed21379bdb0db016d
[1] https://cgit.git.savannah.gnu.org/cgit/gnulib.git/commit/?id=0b97ffdf32bdab909d02449043447237273df75e
If a journal file contains a truncated ZSTD frame (i.e. a frame with
Frame_Content_Size > 0, but with not enough data in Data_Block),
ZSTD_decompressStream() would return a non-zero, non-error value. This
would then skip the error path in the ZSTD_isError() branch and we'd hit
the following assert:
$ build-local/journalctl -o cat --file zstd-truncated.journal
Assertion 'output.pos >= prefix_len + 1' failed at src/basic/compress.c:1236, function decompress_startswith_zstd(). Aborting.
Aborted (core dumped) build-local/journalctl -o cat --file zstd-truncated.journal
Let's handle this situation gracefully and return EBADMSG instead.
Also, add another journalctl invocation to the corrupted-journals test
that goes through the sd_journal_get_data() -> decompress_startswith_zstd()
code path which, among other things, covers the issue when run on the
provided journal file.
Extend fake-report-server.py with optional --cert, --key, --port
arguments for TLS support. Add a test case that generates a
self-signed certificate and tests HTTPS upload of metrics and facts.
Also exercise the --header param.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Add a fake HTTP server (fake-report-server.py) that accepts JSON POST
requests and validates the report structure, and test cases in
TEST-74-AUX-UTILS.report.sh that exercise plain HTTP upload of both
metrics and facts.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Ensure only privileged users can call the system scope machined's
APIs that get data out of a machine
Follow-up for 1bd979dddb
Follow-up for 9153b02bb5
Test the async QMP drive pipeline with real QEMU:
Test 1 (multi-drive): launches vmspawn with --image plus two
--extra-drive flags. This exercises multiple fdset allocations,
pipelined blockdev-add commands relying on FIFO ordering, io_uring
retry callbacks, and multiple device_add commands — all fired
without waiting for responses.
Test 2 (ephemeral): launches vmspawn with --image --ephemeral. This
exercises the most complex async path: blockdev-create fires a
background job, JOB_STATUS_CHANGE events are watched via the event
callback, and when the job concludes the deferred continuation fires
the overlay format node + device_add. If the continuation fails, the
root drive is never attached, the kernel panics, and vmspawn exits
without registering — so successful registration proves the pipeline
works.
Both tests use a raw ext4 image with a minimal init (sleep infinity)
and direct kernel boot. No virtiofsd needed.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Add TEST-87-AUX-UTILS-VM.vmspawn.sh that validates the QMP-varlink
bridge end-to-end using a real QEMU instance:
- Launches vmspawn with --directory and --linux for direct kernel boot
(no UEFI firmware or bootable image needed)
- Waits for machine registration with machined
- Verifies varlinkAddress is exposed in Machine.List
- Tests machinectl pause, resume, poweroff
- Exercises MachineInstance varlink interface directly via varlinkctl:
QueryStatus state verification across pause/resume, Pause, Resume
Skipped automatically if vmspawn, QEMU, or a bootable kernel is not
available. Runs as part of TEST-87-AUX-UTILS-VM in the mkosi
integration test suite.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Using the format specifier +%s%6N with GNU date is honored, and only
prints 6 digits of the nanoseconds portion of the seconds since epoch.
The uutils implementation of date does not honor this, and always prints
all 9 digits. This is a known bug[1], but can be worked around by
adapting this test to use nanoseconds instead of microseconds.
[1] https://github.com/uutils/coreutils/issues/11658
Otherwise you run into errors such as:
"""
../meson.build:2899:28: ERROR: File src/test/test-loop-util.c does not exist.
"""
when deleting a file in git without staging the deletion.
Add CpuUsage, MemoryUsage, IOReadBytes, IOReadOperations, and
TasksCurrent in a standalone socket-activated varlink service. These
metrics are gathered from the kernel via cgroup files and PID1's only
role is mapping unit names to cgroup paths — a separate process can
query PID1 once for that mapping and then read the cgroup files
directly, minimizing PID1 involvement.
The new systemd-report-cgroup-metrics service listens at
/run/systemd/report/io.systemd.CGroup and exposes:
- io.systemd.CGroup.CpuUsage
- io.systemd.CGroup.IOReadBytes
- io.systemd.CGroup.IOReadOperations
- io.systemd.CGroup.MemoryUsage (with type=current/available/peak)
- io.systemd.CGroup.TasksCurrent
This is spun out of #41078 and based on top of it. Will rebase once
that's merged.
Add CpuUsage, MemoryUsage, IOReadBytes, IOReadOperations, and
TasksCurrent in a standalone socket-activated varlink service.
The new systemd-report-cgroup service listens at
/run/systemd/report/io.systemd.CGroup and exposes:
- io.systemd.CGroup.CpuUsage
- io.systemd.CGroup.IOReadBytes
- io.systemd.CGroup.IOReadOperations
- io.systemd.CGroup.MemoryUsage (with type=current/available/peak)
- io.systemd.CGroup.TasksCurrent
Move the push-based streaming compression API from import-compress.c
into compress.c and delete import-compress.c/h. This consolidates all
compression code in one place and makes all compression libraries
(liblzma, liblz4, libzstd, libz, libbz2) runtime-loaded via dlopen
instead of directly linked.
Introduce opaque Compressor/Decompressor types backed by a heap-
allocated struct defined only in compress.c, keeping all third-party
library headers out of compress.h.
Rewrite the per-codec fd-to-fd stream functions as thin wrappers around
the push API via generic compress_stream()/decompress_stream() taking a
Compression type parameter. Integrate LZ4 into this framework using the
LZ4 Frame API, eliminating all LZ4 special-casing.
Extend the Compression enum with COMPRESSION_GZIP and COMPRESSION_BZIP2
and add the corresponding blob, startswith, and stream functions for
both.
Rename the ImportCompress types and functions: ImportCompressType becomes
the existing Compression enum, ImportCompress becomes Compressor (with
Decompressor typedef), and all import_compress_*/import_uncompress_*
become compressor_*/decompressor_*. Rename dlopen_lzma() to dlopen_xz()
for consistency. Make compression_to_string() return lowercase by
default.
Add INT_MAX/UINT_MAX overflow checks for LZ4, zlib, and bzip2 blob
functions where the codec API uses narrower integer types than our
uint64_t parameters.
Migrate test-compress.c and test-compress-benchmark.c to the TEST()
macro framework, new assertion macros, and codec-generic loops instead
of per-codec duplication.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
With the new protocol upgrade support in varlinkctl client we can
now do the equivalent for the server side. This commit adds a new
`serve` verb that will serve any command that speaks stdin/stdout
via varlink and its protocol upgrade feature. This is the
"inetd for varlink".
This is useful for various reasons:
1. Allows to e.g. provide a heavily sandboxed io.myorg.xz.Decompress
varlink endpoint, c.f. xz CVE-2024-3094)
2. Allow sftp over varlink which is quite useful with the
varlink-http-bridge (that has more flexible auth mechanism than
plain sftp).
3. Makes testing the varlinkctl client protocol upgrade simpler.
4. Because we can.
Teach swap units to support the _netdev option as well, which should
make swaps on iSCSI possible. This mirrors the logic we already have for
regular mounts in both the fstab-generator and the core
(mount.c/swap.c).
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Rename nspawn's --user=NAME option to --uid=NAME for selecting the
container user. The -u short option is preserved. --user=NAME and
--user NAME are still accepted but emit a deprecation warning. A
pre-parsing step stitches the space-separated --user NAME form into
--user=NAME before getopt sees it, preserving backwards compatibility
despite --user now being an optional_argument.
Repurpose --user (without argument) and --system as standalone
switches for selecting the runtime scope (user vs system service
manager).
Replace all uses of the arg_privileged boolean with
arg_runtime_scope comparisons throughout nspawn. The default scope
is auto-detected from the effective UID.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
TEST-74-AUX-UTILS has a number of subtests.
test/units/TEST-74-AUX-UTILS.capsule.sh runs first and starts and stops
capsule@foobar.service. Looking at the test, the unit is cleanly stopped.
But later test/units/TEST-74-AUX-UTILS.machine-id-setup.sh tests for
failed units. capsule@foobar.service is listed as failed, causing the
second subtest to fail.
Add the same test in test/units/TEST-74-AUX-UTILS.capsule.sh to see
if the failed test really originates from there.
Lennart Poettering
Yaping Li
Michael Vogt
Luca Boccassi
Frantisek Sumsal
Zbigniew Jędrzejewski-Szmek
Daan De Meyer
Philip Withnall
Ivan Kruglov
Jonathan Davies
Christian Brauner
Nick Rosbrook