This time, various commands with non-trivial requirements are handled.
So the series is shorter, but has a pattern of feature additions and
then those new features being used. The way that --help output is
formatted is changed a bit in various places. Those changes are
described in individual commits.
22 'static int help()' functions left after this.
Almost all programs are simple and have only a single command. When
printing options, let's just use the first COMMAND block found by default.
This means we can drop the explicit program-name argument and reduce
copy&paste boilerplate.
nsresourced hands out transient, recyclable UID/GID ranges to
unprivileged user namespaces, and a BPF-LSM policy is what keeps a
client from leaving inodes owned by its transient range on a file system
that outlives the namespace, so that a later client which gets the same
range can't inherit them.
The old policy decided based on the mount a write went through, which is
forgeable: unshare(CLONE_NEWNS) gives you clones of every mount with
fresh mount IDs but the same superblock, and the mount namespace's owner
is yours to pick. overlayfs sidesteps it entirely. So this reworks the
check to look at where the write actually lands on disk.
The first commit (Christian Brauner's) replaces the per-mount allowlist
with two questions answered from fields the client can't forge and that
survive mount cloning: does the file system die together with the user
namespace (sb->s_user_ns at or below the managed one), and does the
transient ID actually reach the disk, or does an idmapped mount
translate it into something else first. It walks the userns chain
upwards so a nested namespace can't escape it. The map layout changed,
so the pins move to userns-restrict.v2 and the old ones are removed on
install.
The second commit handles overlayfs. A write through an overlay creates
the file on the upper layer via vfs_create(), which only trips the inode
LSM hooks, not the path hooks we attach to, and the one path hook that
does fire sees the overlay mount, which belongs to the managed namespace
and so looks like it dies with it even though the upper may be
persistent. So we judge the upper at mount time in sb_kern_mount and
refuse a writable overlay unless its upper dies with the namespace too.
Reading overlayfs' private ovl_fs/ovl_layer uses CO-RE type flavors
guarded by bpf_core_type_exists(), and the service now orders after
modprobe@overlay.service.
The third commit deals with restarts. The managed set only lives in the
pinned maps, so an upgrade that changes the map layout comes up with
empty maps and every already-running namespace silently loses its
restriction. manager_startup() now re-adds the live registry entries to
the maps in the same pass that reaps the dead ones. The setgroups()
denial is restored from a flag we now store in the registry.
One consequence: a managed namespace can only mount a writable overlay
whose upper dies with it, e.g. a tmpfs it set up itself. If you need a
writable persistent upperdir (mkosi does), use a plain unprivileged user
namespace for that, not a transient nsresourced range.
Add a second COMMAND() for the mount.ddi helper namespace, like for
mount.storage and mount.mstack, so that it is exposed by --introspect-cli
too.
Fixes 1a54ce154c +
66246c98b5.
Before this series, many help() functions opened the pager. This got lost
in the translation. Starting the pager requires access to arg_pager_flags.
While we could pipe this through in case when command_print_help() is
called directly, but this hard in the case when verb_help_auto() or
VERB_COMMON_HELP_AUTO/VERB_COMMON_HELP_AUTO_HIDDEN are used. Let's instead
extend the (const static) CommandDescription structure with a pointer to
the arg_pager_flags variable. This restores the pager functionality in a
uniform fashion with a one-line change in the individual binaries.
tar_c() keeps a database of already seen inodes so it can emit hardlinks
rather than storing the same content repeatedly. It prefers to keep that
database on a tmpfs superblock of its own, which is created lazily on the
first hardlinkable inode and which requires CAP_SYS_ADMIN in the caller's
user namespace. Failing that it falls back to a temporary directory under
/var/tmp/.
In the tar child forked off by import_fork_tar_c() the tmpfs branch can
never succeed: the child joins the user namespace, then drops all
capabilities but CAP_DAC_OVERRIDE, and only afterwards runs tar_c(), by
which point CAP_SYS_ADMIN is long gone. Every export therefore silently
took the /var/tmp/ fallback.
That fallback is a problem beyond being the slower path. The child runs
with the transient UID range assigned to the user namespace, so the
directory it creates is owned by a recyclable UID on a file system that
outlives the namespace, which is exactly what the nsresourced BPF-LSM
policy exists to prevent. With that policy in effect the mkdtemp() is
refused and the export fails with EPERM.
Create the database up front, in the window after the user namespace has
been joined and before the capabilities are dropped, and pass the
resulting fd into tar_c(). The database then lives on a superblock that
dies together with the namespace, no persistent file system is touched,
and the sandbox around tar is unchanged.
tar_c() keeps its lazy fallback for the callers that do not pass an fd,
i.e. systemd-dissect and the test, which do not run under a transient
range. The fd is borrowed, not consumed.
Signed-off-by: Daan De Meyer <daan@amutable.com>
Use the image metadata size when building JSON output, matching the
value already shown by the text output.
Reproducer: systemd-dissect --json=short image.raw
Before: text output showed Size, but JSON omitted the top-level size
field.
Follow-up: be5bee2a13
Since 4c0d8d9673, most dlopen notes are
set at the beginning of the executables. Let's manage all dlopen notes
there, rather than setting them where dlopen is called.
Note that the only exceptions are the LIBBPF_NOTE for networkd and
nsresource. Since dlopen_bpf() is wrapped in an `#if` guard, the notes
are instead set within the corresponding functionality.
As a result, the DLOPEN_FOO() wrapper macros are no longer needed and
can be dropped completely.
Reflinking is a safe optimization whenever the source and destination are
regular, seekable files on a filesystem that supports cloning. Requiring each
caller to opt in through COPY_REFLINK adds flag plumbing without changing the
necessary fallback behavior.
Drop COPY_REFLINK, renumber the remaining flags, and have copy_bytes_full()
unconditionally try FICLONE or FICLONERANGE before entering its normal copy
loop. Isolate the clone attempt and its file-offset bookkeeping in a helper so
copy_bytes_full() only has to handle its cloned, unavailable, and error results.
A successful clone therefore completes in one operation, while an unsupported
or rejected clone falls back to progress-limited copy_file_range(), sendfile(),
or buffered copying. Keep the public reflink helpers for operations that
specifically require clone semantics.
Existing test-copy coverage exercises both bounded and unbounded regular-file
copies through copy_bytes_full().
Signed-off-by: Daan De Meyer <daan@amutable.com>
Previously, when a library was dynamically loaded via a helper function
inside libsystemd-shared.so, the resulting dlopen ELF note was not
propagated to the invoking executable's ELF metadata.
This commit explicitly annotates each executable with the relevant
dlopen ELF notes for any optional dependencies it might potentially
load. This ensures that package managers and build systems can properly
discover runtime dependencies that are triggered indirectly.
This also switches all these notes to be defined via SD_ELF_NOTE_DLOPEN_ANCHORED().
Consequently, any notes added within unreachable or unused functions will be
automatically garbage-collected by the linker (--gc-sections) instead of
bloating the final binary.
E.g. unused p11-kit library dependency is now dropped from
systemd-repart.standalone binary.
Before:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
idn Support for internationalized domain names libidn2.so.0 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
p11-kit Support for PKCS11 hardware tokens libp11-kit.so.0 suggested
```
After:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
```
When copying a directory into an image, open an existing destination
with O_NOFOLLOW before passing it to copy_tree_at(). This rejects a
symlink used as the final destination component without a separate
stat-before-use check.
Previously the pre-opened destination followed that symlink. That
allowed --copy-to to be redirected outside the image root when the
image was a directory tree.
If the destination does not exist yet, keep delegating creation to
copy_tree_at(). Real existing directories still use COPY_MERGE through
the opened directory fd.
Signed-off-by: dongshengyuan <dongshengyuan@uniontech.com>
This makes each `foo_cflags` dependency define a `SYSTEMD_CFLAGS_MARKER_FOO`
macro, and checks if the macro is set when headers provided by external
libraries are included.
With this, we can fail fast at compile time if necessary `_cflags`
dependencies are omitted in meson.build. Missing dependencies found by this
mechanism have been added across the tree.
663f0bf5cb stopped reusing the original block device fd whenever
partition scanning was requested (LO_FLAGS_PARTSCAN) but couldn't be
enabled on the device, so that nested partition tables on devices the
kernel won't scan (e.g. the pmOS/android case) get exposed via a real
loop device.
However that also forced a pointless loop device for any partition that
carries a file system directly, e.g. a btrfs subvolume mounted via
MountImages=. For multi-device btrfs this is fatal: the kernel rejects
seeing the same member via both the original partition and the loop
device, and the mount fails.
A loop device is only ever needed here to expose a nested partition
table. So only refuse the shortcut when the device actually carries one,
probed via gpt_probe(), instead of whenever partition scanning is
disabled. Devices carrying a file system directly (or nothing) take the
shortcut as before.
Add an integration test to cover the failure scenario of the original
issue.
Fixes: https://github.com/systemd/systemd/issues/42520
Replaces: https://github.com/systemd/systemd/pull/42576
Follow-up for 663f0bf5cb79ecaf6dd71441ecdc9dc401e7eae6
Co-Authored-By: Luca Boccassi <luca.boccassi@gmail.com>
Co-developed-by: Claude Opus 4.8 <noreply@anthropic.com>
Switch the executable-owned call sites from the plain dlopen_<lib>()
helpers to the new DLOPEN_<LIB>() macros, so that each binary that loads
an optional library at runtime now carries its own .note.dlopen note
instead of relying solely on libsystemd-shared. Each call site passes a
priority reflecting whether it hard-depends on the library (required) or
degrades gracefully without it (recommended).
Co-developed-by: Claude Opus 4.8 <noreply@anthropic.com>
I am really not a fan of full code lines passed to macros as parameters.
Let's get rid of the 3rd parameter of FOREACH_OPTION() hence:
1. Let's return errors just as a regular value (though a negative one),
that can be handled via a OPTION_ERROR case statement for the switch.
This normalizes handling of the error, just like any other event
returned by the option parser.
2. In order to avoid exploding the amount of boilerplate in each use
(that just propagates the error on OPTION_ERROR), let's then
introduce an explicit FOREACH_OPTION_OR_RETURN(), that returns from
the calling function on its own (and makes that clear in the name).
Together this cleans up, normalizes the logic and shortens the code.
It was requested to make the 'c', 'opt', and 'arg' params the same, i.e.
defined through the FOREACH_OPTION macro. But we can't do that easily,
because 'c' was defined in the for loop definition, and we can only
define variables of the same type in that way. Also, in some cases we
need only 'c', in other cases with need 'c' and 'arg, in some cases 'c'
and 'opt', and in other cases all three. We'd need to either
conditionalize or mark those variables with _unused_ to deal with
compiler warnings. But a different approach works quite nicely: add
state.opt and state.arg to show the current option and it's argument.
(The short names are picked on purpose to reduce verbosity since those
are used a lot.)
Every dlopen_xxx() helper now takes an int log_level argument. It is
passed through to dlopen_many_sym_or_warn() (which in turn propagates it
to dlopen_verbose() for the library-not-installed case), and is used by
the fallback stub when support for the library is not compiled in to
emit a "<lib> support is not compiled in." message at the caller's level.
Callers pass LOG_DEBUG when gracefully degrading, or a higher level when
the failure should surface, and no longer need to log redundantly at the
call site.
As part of this, dlopen_bpf_full() (which already took a log_level) is
merged into dlopen_bpf() rather than keeping both.
The static inline fallbacks used to live in the headers, which required
pulling log.h in from every header that declared a dlopen_xxx(). Move
them into the .c files instead: the declaration is always outside the
#if HAVE_XXX block, the impl sits outside the outer #if HAVE_XXX wrap
with its own internal #if HAVE_XXX/#else/#endif, and apparmor-util.c,
idn-util.c, libmount-util.c and pam-util.c are now always compiled so
they can host their stubs.
When a udev rule uses ENV{SYSTEMD_WANTS}+="systemd-loop@.service" on a
block device, the %f specifier in the service file resolves to the sysfs
path rather than the device node path. Detect sysfs paths in
parse_image_path_argument() and resolve them to the corresponding
devnode using sd_device_new_from_syspath() + sd_device_get_devname().
We generally want to print an error message if table_print()
fails. Add a helper function for this and use it consistently.
This does one of the three things depending on the call site:
- a no-change reformatting of the code
- change from a custom message to the generic one
- addition of the error message where previously none was printed
In the third case, the actual use impact is very small, since the
table formatting is very unlikely to fail. But if it did, we would
often return an error without any message whatsoever, which we
never want to do.
Almost all callers of table_print() specify stdout or NULL (equivalent
to stdout) as the output stream. Simplify things by not requiring the
the stream to be specified.
In almost all cases, the printing of the table is surrounded by normal
printfs() that don't do explicit flushing and for which we don't check
the output stream status. Let's simplify most callers and skip this
step. The reason is not so much to avoid the extra step itself, but
instead to avoid the _handling_ of the potential failure. We generally
only want to print an error message for ENOMEM and other "internal"
errors, so strictly speaking we should filter out the errors from the
stream. By skipping the flush&check step we implicitly do this.
When callers do not explicitly request read-only mode, pass open_flags
as -1 (auto-detect) instead of hardcoding O_RDWR. This enables the
existing O_RDWR-to-O_RDONLY retry logic in loop_device_make_by_path_at()
which falls back to O_RDONLY when opening the backing device with O_RDWR
fails with EROFS or similar errors.
Previously, callers passed O_RDWR explicitly when read-only mode was not
requested, which bypassed the retry logic entirely. This meant that
inherently read-only block devices (such as CD-ROMs) would fail to open
instead of gracefully falling back to read-only mode.
Also propagate the unresolved open_flags through
loop_device_make_by_path_at() into loop_device_make_internal() instead
of resolving it to O_RDWR early. For loop_device_make_by_path_memory(),
resolve to O_RDWR immediately since memfds are always writable.
In mstack, switch from loop_device_make() to
loop_device_make_by_path_at() with a NULL path, which reopens the
O_PATH file descriptor with the appropriate access mode. This is
necessary because the backing file descriptor is opened with O_PATH,
which prevents loop_device_make_internal() from auto-detecting the
access mode via fcntl(F_GETFL).
The column index is moved to the first position. I think we're unlikely
to want to synchronize widths of *different* columns, and having just
one column argument makes the callers simpler. Also, the type is changed
to size_t to match other functions, and this avoids the need to cast
to size_t in the callers.
After writing the code to parse options in a bunch of places, I think
passing the argc+argv to various functions to query state is annoying.
It seems nicer to just stash them in the state struct once.
This feel a bit like a hack, but it works OK. The width of the first
column of verbs or options in different sections is measured and
applied to the other tables. This makes the second column aligned.
clang says:
src/dissect/dissect.c:292:17: warning: label followed by a declaration is a C23 extension [-Wc23-extensions]
Another option would be to raise the C standard to C23. I think that'd
make sense, but it's better to do it separately.
Some cosmetic differences in descriptions are made.
--quiet was miscategorized as a command (172fadda65),
--usr-hash and --usr-hash-sig were not listed (10b8d65f3f).
ENOENT means /dev/loop-control isn't there which means we're in a
container and should go via mountfsd.
At the same time, reverse the check for fatal actions as almost all
actions can be done via mountfsd, only --attach needs the loop device.
I expressed the issue I have with parse_tristate_argument()
in #37751: it doesn't add any value to direct use of parse_tristate();
on the contrary, it doesn't support means to reset the arg to "auto"/-1 state.
The mere reason it existed is that we need a int type ret param.
Since the previous attempt to address this mess failed, let's
try to make the function more useful by making it accept "auto".
I figure this is useful on its own.
As requested in
https://github.com/systemd/systemd/pull/40652#discussion_r2831833996,
the function name is suffixed with _with_auto() to establish
that "auto" is handled internally.
So far we opened a new Varlink connection for every mountfsd/nsresourced
method call. Given each tool only does a very small number of calls
(usually 1…5) on them and the connections are cheap this is not too
wasteful. Nonetheless, let's do something about it, and allow reusing
the connection for multiple calls.
This not only makes things a bit more efficient, but has one more
important benefit: Varlink connections pin the security context of the
client when connecting. This means that varlink method calls done with a
connection established while some code was privileged will still operate
as privieged once privs are dropped, until the connection is closed.
This pinning effect is really nice, as it gives us behaviour in a
"capability system" like scheme. Later code is going to use that to
continue doing certain priv userns ops even after unsharing userns and
becoming fully unpriv.
Currently, if we're copying a file, we won't copy the owner UID/GID
from the source. If we're copying a directory, we will copy the owner
UID/GID from the source. Let's give users a bit more control over this
behavior by introducing --copy-ownership= which will default to the
current behavior but allows users to explicitly enable/disable copying
of ownership.
Let's check for CAP_SYS_ADMIN instead of root for these, and make
unmounting more graceful if we can't access the backing loop device
because of permission issues. This allows mounting and unmounting images
from an unprvileged mount namespace. The actual files in the image will
end up owned by nobody:nobody because we'll be in an unprivileged user
namespace, but assuming the directory permissions are not too strict, this
still allows interacting with the image in useful ways.
These actions interact with the host. The former needs privileges to
write into the image, the latter needs privileges to write on the host.
Neither will have the privileges required if the image is attached under
a private userns, hence, don't use one.
RootImageOptions=/ExtensionImages=/MountImages= all support custom
mount options, but mountfsd does not support it. Add varlink
parameters to allow callers to specify mount options so that
those directives can work as expected. Require additional privs via
polkit.
Currently, pick_filter_image_any matches any image
with any suffix, which is way more than it should
be doing. It should only be matching images with
the .raw suffix.
Let's address this shortcoming by allowing to pass
multiple filters to path_pick(), and define
pick_filter_image_any as the combination of the
raw and directory image filters.
Fixes#40083
RootImageOptions=/ExtensionImages=/MountImages= all support
custom mount options, use the new mountfsd parameters to
configure them if they are specified.
This requires additioanl privileges via polkit due to security
implications of mount options, so document an example policy
that allows to use the nosuid mount option.
felix h
Zbigniew Jędrzejewski-Szmek
Luca Boccassi
Daan De Meyer
Yu Watanabe
dongshengyuan
Lennart Poettering
Mike Yuan
DaanDeMeyer
Luca Boccassi