Drop the -fundamental suffix from src/fundamental/ headers in favor of names
that match their src/basic/ or src/shared/ counterparts (e.g.
macro-fundamental.h -> macro.h, assert-fundamental.h -> assert-util.h,
cleanup-fundamental.h -> cleanup-util.h). Rename src/basic/{btrfs,label}.{c,h}
to use the -util suffix to match the existing shared/btrfs-util and
shared/label-util siblings. Rename src/shared/mkdir-label.{c,h} to mkdir.{c,h}
and src/shared/tmpfile-util-label.{c,h} to tmpfile-util.{c,h} to match the
corresponding src/basic names.
This saves us from having to come up with separate names for files that do
the same thing across tiers, and it makes it easier to move stuff between
src/fundamental, src/basic and src/shared: consumers just #include "foo.h"
and pick up whichever tier their -I path resolves to first, so call sites
don't need to be updated when an API moves between layers.
Where a higher-tier wrapper exists (e.g. src/basic/macro.h wrapping
src/fundamental/macro.h), the wrapper uses an explicit "../fundamental/foo.h"
or "../basic/foo.h" relative include for the lower-tier header. We can't use
GCC's #include_next directive for this — when the wrapper is reachable both
via same-dir-as-source lookup and via -I (e.g. -Isrc/shared) for the
directory it lives in, #include_next advances by exactly one slot in libcpp's
internal directory chain and lands on the same physical directory it was
already in, never reaching the lower-tier sibling (see make_cpp_dir() in
gcc/libcpp/files.cc:1986).
To make sure the right headers are always picked up, the include directories
are reordered so that e.g. src/shared always takes priority over src/basic and
similar for the other directories.
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
When there is a NOTIFY_SOCKET, and FDs are received from the
payload following the FD Store protocol, forward them up the
chain to the service manager that is managing nspawn.
This allows FD Store persistence across container restarts,
and can chain up for user managers as well to survive restarting
those, or reexecs, and in the future reboots too via LUO.
Add a new test case to exercise the PID1 -> user session -> nspawn -> payload
chain.
When the kernel hands the command line to PID 1, any KEY=VALUE assignment
whose KEY does not contain a '.' is exported as an environment variable
(with '-' replaced by '_') rather than passed as an argument. Mimic the
same split in --boot mode so kernel-cmdline-style arguments passed after
the container path behave as they would on a real boot.
I am really not a fan of full code lines passed to macros as parameters.
Let's get rid of the 3rd parameter of FOREACH_OPTION() hence:
1. Let's return errors just as a regular value (though a negative one),
that can be handled via a OPTION_ERROR case statement for the switch.
This normalizes handling of the error, just like any other event
returned by the option parser.
2. In order to avoid exploding the amount of boilerplate in each use
(that just propagates the error on OPTION_ERROR), let's then
introduce an explicit FOREACH_OPTION_OR_RETURN(), that returns from
the calling function on its own (and makes that clear in the name).
Together this cleans up, normalizes the logic and shortens the code.
DEFINE_POINTER_ARRAY_CLEAR_FUNC() generates a helper of the form
helper_array_clear(T *array, size_t n) that drops each element but does
not free the array itself, parallel to DEFINE_POINTER_ARRAY_FREE_FUNC()
for cases where the array has automatic storage duration.
CLEANUP_ELEMENTS() pairs with these helpers to provide a _cleanup_-like
attribute for fixed-size arrays: the bound is taken from ELEMENTSOF(),
and the helper is invoked across the elements at scope exit. Compared to
CLEANUP_ARRAY(), the storage is neither freed nor zeroed.
Migrate various logic across the tree over to the new macros.
sd-device: use DEFINE_POINTER_ARRAY_CLEAR_FUNC() for sd_device_unref_array_clear()
Replace the local device_unref_many() helper with the macro-generated
equivalent.
format-table: switch help-table arrays to CLEANUP_ELEMENTS()
Generate table_unref_array_clear() via DEFINE_POINTER_ARRAY_CLEAR_FUNC()
and convert the help-table arrays in bootctl, cryptenroll, nspawn,
repart and vmspawn to CLEANUP_ELEMENTS(). The arrays no longer need a
trailing NULL slot, so the size matches ELEMENTSOF() of the groups
array.
firewall-util: switch netlink message arrays to CLEANUP_ELEMENTS()
Generate sd_netlink_message_unref_array_clear() via
DEFINE_POINTER_ARRAY_CLEAR_FUNC() in place of the NULL-terminated
sd_netlink_message_unref_many(), and convert the two stack arrays of
sd_netlink_message pointers to CLEANUP_ELEMENTS().
option_parser_next_arg() is renamed to option_parser_peek_next_arg()
to match option_parser_consume_next_arg().
A new helper is added option_parser_get_arg(…, n). It is a common pattern
to only need a single arg, and getting an array and extracting a single
item from it is too verbose.
It was requested to make the 'c', 'opt', and 'arg' params the same, i.e.
defined through the FOREACH_OPTION macro. But we can't do that easily,
because 'c' was defined in the for loop definition, and we can only
define variables of the same type in that way. Also, in some cases we
need only 'c', in other cases with need 'c' and 'arg, in some cases 'c'
and 'opt', and in other cases all three. We'd need to either
conditionalize or mark those variables with _unused_ to deal with
compiler warnings. But a different approach works quite nicely: add
state.opt and state.arg to show the current option and it's argument.
(The short names are picked on purpose to reduce verbosity since those
are used a lot.)
Add --forward-journal=FILE|DIR to forward the container's journal
entries to the host via systemd-journal-remote. When specified,
nspawn starts systemd-journal-remote listening on a Unix socket,
bind-mounts it into the container at /run/host/journal/socket, and
passes a journal.forward_to_socket credential pointing to it.
Add --forward-journal-max-use=, --forward-journal-keep-free=,
--forward-journal-max-file-size=, and --forward-journal-max-files=
to configure disk usage limits for the forwarded journal.
Consolidate nspawn's per-machine on-disk state under a single runtime
directory at /run/systemd/nspawn/<machine>/. The container rootdir
mount point moves from /tmp/nspawn-root-XXXXXX to <runtime_dir>/root,
the unix-export directory moves from
/run/systemd/nspawn/unix-export/<machine> to <runtime_dir>/unix-export,
and the journal-remote socket lives at
<runtime_dir>/journal-remote-socket. Update ssh-generator and
ssh-proxy to follow the new unix-export path layout.
Extract fork_journal_remote() into fork-notify.{c,h} as a shared
helper used by both nspawn and vmspawn, replacing vmspawn's
start_systemd_journal_remote().
Extract runtime_directory_make() into path-lookup.{c,h} as a shared
helper used by both nspawn and vmspawn, replacing vmspawn's inline
runtime directory creation logic.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Every dlopen_xxx() helper now takes an int log_level argument. It is
passed through to dlopen_many_sym_or_warn() (which in turn propagates it
to dlopen_verbose() for the library-not-installed case), and is used by
the fallback stub when support for the library is not compiled in to
emit a "<lib> support is not compiled in." message at the caller's level.
Callers pass LOG_DEBUG when gracefully degrading, or a higher level when
the failure should surface, and no longer need to log redundantly at the
call site.
As part of this, dlopen_bpf_full() (which already took a log_level) is
merged into dlopen_bpf() rather than keeping both.
The static inline fallbacks used to live in the headers, which required
pulling log.h in from every header that declared a dlopen_xxx(). Move
them into the .c files instead: the declaration is always outside the
#if HAVE_XXX block, the impl sits outside the outer #if HAVE_XXX wrap
with its own internal #if HAVE_XXX/#else/#endif, and apparmor-util.c,
idn-util.c, libmount-util.c and pam-util.c are now always compiled so
they can host their stubs.
In nspawn.c's run_container() the child_netns_fd = receive_one_fd(...)
failure path logged 'r' instead of the negative errno returned in
child_netns_fd, so the actual error from receive_one_fd was being
overwritten by whatever 'r' happened to hold. The other receive_one_fd
call sites in the same function use the returned fd variable directly
(mntns_fd, etc.), so align this one.
In shared/nsresource.c's nsresource_add_cgroup() the cgroup_fd_idx =
sd_varlink_push_dup_fd(...) failure path logged userns_fd_idx, which
is the previous successful push's index, not the negative errno we
just got from pushing cgroup_fd. Log cgroup_fd_idx instead.
Both were flagged by static analysis (#41709) and match the immediately
preceding userns_fd-path pattern that was presumably copy-pasted.
Refs #41709.
If range is NULL (i.e. when PrivateUsers= doesn't contain ':'),
both later error paths will then pass NULL to log_syntax():
~# cat foo.nspawn
[Exec]
PrivateUsers=9999999999999999999
~# SYSTEMD_LOG_LEVEL=debug systemd-nspawn -D foo |& grep foo.nspawn
Found settings file: /root/foo.nspawn
/root/foo.nspawn:2: UID/GID shift invalid, ignoring: (null)
or
~# cat foo.nspawn
[Exec]
PrivateUsers=4294967294
~ # SYSTEMD_LOG_LEVEL=debug systemd-nspawn -D foo |& grep foo.nspawn
Found settings file: /root/foo.nspawn
/root/foo.nspawn:2: UID/GID shift and range combination invalid, ignoring: (null)
Let's just use rvalue in both of these cases instead.
This will make it easier to add new modes of operation later.
But I'm happy with how this came out — I think the mode setting
is nicer to read then the old bool.
I promised in https://github.com/systemd/systemd/pull/41426 that its the
final update for coccinelle pointer deref checks. However it turned out
there is this coccinelle/parsing_hacks.h that I wasn't aware of. The
file missed some important things like _cleanup_(x) that prevented
coccinelle to check a bunch of functions.
This PR adds some missing defines to the parsing_hacks.h and fixes the
missing asserts(). I apologize that its a bit long (and frankly boring)
and that I missed this earlier.
The last commit contains one small behavior change (ret in
sd_varlink_idl_parse() is now really optional) but the big one is very
mechanical.
Add a new --restrict-address-families= command line option and
corresponding RestrictAddressFamilies= setting for .nspawn files to
restrict which socket address families may be used inside a container.
Many address families such as AF_VSOCK and AF_NETLINK are not
network-namespaced, so restricting access to them in containers
improves isolation. The option supports allowlist and denylist modes
(via ~ prefix), as well as "none" to block all families, matching the
semantics of RestrictAddressFamilies= in unit files.
The address family parsing logic is extracted into a shared
parse_address_families() helper in parse-helpers.c, which is now also
used by config_parse_address_families() in load-fragment.c.
This is currently opt-in. In a future version, the default will be
changed to restrict address families to AF_INET, AF_INET6 and AF_UNIX.
The updated parsing_hacks.h file uncovered a bunch of extra
things that the check-pointer-deref coccinelle script flags.
This commit fixes them to make the tree check-pointer-deref clean.
This is a helper macro that defines a function to drop elements of an
array but not the array itself. I used the "_many" suffix because it
most closely matches what happens here: we are calling the cleanup
function a bunch of times.
Uses stop_at_first_nonoption for POSIX-style option parsing.
Includes a fixup for b4df0a9ee6, where
global optarg was used instead of the function param. This made no
difference previously because they were always equal.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Bundle scope, buses, and registration success booleans into a
MachineRegistrationContext struct. This eliminates the reterr_registered_system and
reterr_registered_user output parameters from
register_machine_with_fallback_and_log() and the corresponding input
parameters from unregister_machine_with_fallback_and_log().
The struct carries state from registration to unregistration so the
caller no longer needs to manually thread individual booleans between
the two calls.
register_machine_with_fallback_and_log() goes from 7 to 3 parameters,
unregister_machine_with_fallback_and_log() goes from 5 to 2.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Replace the long positional parameter lists in register_machine() and
register_machine_with_fallback_and_log() with a MachineRegistration
struct that bundles all machine-describing fields.
This reduces register_machine() from 13 parameters to 3 and
register_machine_with_fallback_and_log() from 17 parameters to 7.
Callers now use designated initializers, which makes omitted fields
(zero/NULL/false) implicit and the code much more readable.
Field names are aligned with the existing Machine struct in machine.h
(id, root_directory, vsock_cid, ssh_address, ssh_private_key_path).
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Rename nspawn's --user=NAME option to --uid=NAME for selecting the
container user. The -u short option is preserved. --user=NAME and
--user NAME are still accepted but emit a deprecation warning. A
pre-parsing step stitches the space-separated --user NAME form into
--user=NAME before getopt sees it, preserving backwards compatibility
despite --user now being an optional_argument.
Repurpose --user (without argument) and --system as standalone
switches for selecting the runtime scope (user vs system service
manager).
Replace all uses of the arg_privileged boolean with
arg_runtime_scope comparisons throughout nspawn. The default scope
is auto-detected from the effective UID.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Move register_machine() and unregister_machine() from
vmspawn-register.{c,h} into shared machine-register.{c,h} so both
nspawn and vmspawn can use the same implementation.
The unified register_machine() uses varlink first (for richer
features like SSH support and unit allocation) with a D-Bus
RegisterMachineWithNetwork fallback for older machined. The
interface adds a class parameter ("vm" or "container") and
local_ifindex for nspawn's network interface support.
The unified unregister_machine() similarly tries varlink first
(io.systemd.Machine.Unregister) before falling back to D-Bus.
Both register_machine() and unregister_machine() only log at debug
level internally, leaving error/notice logging to callers.
Add register_machine_with_fallback() which tries system and/or user
scope registration based on a RuntimeScope parameter
(_RUNTIME_SCOPE_INVALID for both), and
unregister_machine_with_fallback() as its counterpart. Both use
RET_GATHER() to collect errors from each scope.
Make --register= a tristate (yes/no/auto) defaulting to auto. When
set to auto, registration failures are logged at notice level and
ignored. When set to yes, failures are fatal.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Follow-up for af5126568a ("nspawn: keep backing files for boot_id and
kmsg bind mounts alive").
The backing files for the boot_id and kmsg bind mounts were previously
created in the inner child. However, /run/host/ is remounted read-only
by mount_all() in the inner child (via the MOUNT_IN_USERNS mount table
entry) before setup_boot_id() and setup_kmsg() run, so creating files
there would fail with EROFS.
Fix this by splitting the file creation into separate functions
(setup_boot_id_file() and setup_kmsg_fifo()) that run in the outer
child, where /run/host/ is still writable. The bind mounts onto /proc
remain in the inner child, since procfs is only mounted there.
Also move the backing files from /run/ to /run/host/ and drop the dot
prefix, since /run/host/ is the container-manager-owned namespace and
there is no need to hide these files there. Additionally, apply
userns_lchown() to the created files, matching the convention used by
all other outer child functions that create files in the container.
Both setup_boot_id() and setup_kmsg() previously created temporary files
in /run, bind mounted them over their respective /proc targets, and then
immediately unlinked the backing files. While the bind mount keeps the
inode alive, the kernel marks the dentry as deleted.
This is a problem because bind mounts backed by unlinked files cannot be
replicated: both the old mount API (mount(MS_BIND)) and the new mount
API (open_tree(OPEN_TREE_CLONE) + move_mount()) fail with ENOENT when
the source mount references a deleted dentry. This affects
mount_private_apivfs() in namespace.c, which needs to replicate these
submounts when setting up a fresh /proc instance for services with
ProtectProc= or similar sandboxing options — with an unlinked backing
file, the boot_id submount simply gets lost.
Fix this by using fixed paths (/run/proc-sys-kernel-random-boot-id and
/run/proc-kmsg) instead of randomized tempfiles, and not unlinking them
after the bind mount. The files live in /run which is cleaned up on
shutdown anyway.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When callers do not explicitly request read-only mode, pass open_flags
as -1 (auto-detect) instead of hardcoding O_RDWR. This enables the
existing O_RDWR-to-O_RDONLY retry logic in loop_device_make_by_path_at()
which falls back to O_RDONLY when opening the backing device with O_RDWR
fails with EROFS or similar errors.
Previously, callers passed O_RDWR explicitly when read-only mode was not
requested, which bypassed the retry logic entirely. This meant that
inherently read-only block devices (such as CD-ROMs) would fail to open
instead of gracefully falling back to read-only mode.
Also propagate the unresolved open_flags through
loop_device_make_by_path_at() into loop_device_make_internal() instead
of resolving it to O_RDWR early. For loop_device_make_by_path_memory(),
resolve to O_RDWR immediately since memfds are always writable.
In mstack, switch from loop_device_make() to
loop_device_make_by_path_at() with a NULL path, which reopens the
O_PATH file descriptor with the appropriate access mode. This is
necessary because the backing file descriptor is opened with O_PATH,
which prevents loop_device_make_internal() from auto-detecting the
access mode via fcntl(F_GETFL).
Coverity flags UINT32_MAX - data.container_id as an underflow
when container_id could be UID_INVALID (UINT32_MAX). After
successful sd_json_dispatch_uid_gid(), the values are guaranteed
valid, but Coverity cannot trace through the callback. Add
asserts to document this invariant.
CID#1548072
Follow-up for 91c4d1affd
/run/systemd/inaccessible/ exists only on host - in the container we have
/run/host/inaccessible/, and since all the inaccessible mounts have
MOUNT_IN_USERNS we need to use the latter one, otherwise the masking
gets silently skipped:
~# SYSTEMD_LOG_LEVEL=debug systemd-nspawn -q --directory=foo ls -la /proc/kallsyms
...
Bind-mounting /run/systemd/inaccessible/reg on /proc/kallsyms (MS_BIND "")...
Failed to mount /run/systemd/inaccessible/reg (type n/a) on /proc/kallsyms (MS_BIND ""): No such file or directory
Changing mount flags /proc/kallsyms (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_BIND "")...
Failed to mount n/a (type n/a) on /proc/kallsyms (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_BIND ""): Invalid argument
Bind-mounting /run/systemd/inaccessible/reg on /proc/kcore (MS_BIND "")...
Failed to mount /run/systemd/inaccessible/reg (type n/a) on /proc/kcore (MS_BIND ""): No such file or directory
Changing mount flags /proc/kcore (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_BIND "")...
Failed to mount n/a (type n/a) on /proc/kcore (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_BIND ""): Invalid argument
...
Inner child finished, invoking payload.
-r--r--r--. 1 root root 0 Feb 25 13:19 /proc/kallsyms
We expose this via --private-users-delegate= which takes the number of
ranges to delegate. On top of delegating the ranges, we also mount in
the nsresourced socket and the mountfsd socket so that nested containers
can use nsresourced to allocate from the delegated ranges and mountfsd to
mount images.
Finally, we also create /run/systemd/dissect-root with systemd-tmpfiles to
make sure it is always available as unpriv users won't be able to create it
themselves.
So far we opened a new Varlink connection for every mountfsd/nsresourced
method call. Given each tool only does a very small number of calls
(usually 1…5) on them and the connections are cheap this is not too
wasteful. Nonetheless, let's do something about it, and allow reusing
the connection for multiple calls.
This not only makes things a bit more efficient, but has one more
important benefit: Varlink connections pin the security context of the
client when connecting. This means that varlink method calls done with a
connection established while some code was privileged will still operate
as privieged once privs are dropped, until the connection is closed.
This pinning effect is really nice, as it gives us behaviour in a
"capability system" like scheme. Later code is going to use that to
continue doing certain priv userns ops even after unsharing userns and
becoming fully unpriv.
namespace_enter() now does a is_our_namespace() check, which requires
/proc on older kernels, which is not available anymore after we call
do_wipe_fully_visible_api_fs() in wipe_fully_visible_api_fs().
Let's just call setns() instead as namespace_enter() is overkill to
enter a single namespace anyway.
This patch adds a `--ephemeral` option to vmspawn that will turn
on snapshot mode on the qemu disk passed via `--image` or the
directory passed via `--directory`. For disk images it uses the
native mechanism that qemu provides. For directories it (re)uses
the snapshot mechanism that nspawn is using, i.e. it will create
a btrfs snapshot if possible and if not falls back to a traditional
directory copy.
Frantisek Sumsal
Daan De Meyer
Luca Boccassi
Diego Viola
Lennart Poettering
Zbigniew Jędrzejewski-Szmek
mukunda katta
Daan De Meyer
Michael Vogt
Christian Brauner
Mike Yuan
Thomas Weißschuh
nikstur
Zbigniew Jędrzejewski-Szmek