Turns out that clang's interprocedural analysis is quite smart and can
look through our expand_to_usable() trick, which then causes crashes
with _FORTIFY_SOURCE=3.
clang's interprocedural analysis can see that expand_to_usable() simply
returns its first argument, so it replaces all uses of the return value
with that argument (the original realloc() result). This effectively
bypasses expand_to_usable()'s alloc_size attribute, causing the fortify
check to use the (smaller) size from realloc() instead, which eventually
leads to a false-positive buffer overflow:
$ build/test-varlink-idl
/* test_parse_format */
...
*** buffer overflow detected ***: terminated
Aborted (core dumped) build/test-varlink-idl
This is not an issue with gcc (at least not yet), since gcc sees
expand_to_usable() as an opaque user-defined allocation-like function
and simply trusts the alloc_size attribute that comes with it.
To fix this, let's add a simple no-op barrier to malloc_sizeof_safe()
that clobbers the input pointer, which causes
__builtin_dynamic_object_size() to return (size_t)-1 - this is
interpreted as an "unknown" size by the following fortify check which is
then skipped instead of triggering the assertion.
Similarly, test-alloc-util now doesn't call malloc_usable_size()
directly but instead goes through malloc_sizeof_safe(), so it's also
guarded by the barrier.
This follows the already established pile of similar workaround for the
same class of issues we encountered with gcc, namely [0], which prompted
[1], that was later reverted in [2], and then followed by another couple
of fixes in [3] and [4].
Resolves: #43178
[0] https://github.com/systemd/systemd/issues/22801
[1] https://github.com/systemd/systemd/commit/0bd292567a543d124cd303f7dd61169a209cae64
[2] https://github.com/systemd/systemd/commit/2cfb790391958ada34284290af1f9ab863a515c7
[3] https://github.com/systemd/systemd/commit/7929e180aa47a2692ad4f053afac2857d7198758
[4] https://github.com/systemd/systemd/commit/4f79f545b3c46c358666c9f5f2b384fe50aac4b4
switch_root() calls a blanket sync() before detaching the old root
file system, in order to make sure it is in a good state before it
becomes unreachable via MNT_DETACH/pivot_root().
A global sync() however flushes out *every* mounted file system on
the system, not just the ones we are actually about to detach. On
real-world systems that commonly have several additional mounted file
systems (separate /home, /var, additional data partitions, network
shares, removable media, ...) this needlessly delays switch_root() with
completely unrelated I/O. This matters in particular for
initrd-switch-root.service, which runs this code on the critical path
of pretty much every single boot with an initrd, and for soft-reboot.
Replace the global sync() with a new sync_departing_file_systems()
helper that walks /proc/self/mountinfo and calls syncfs() on every
file system except:
- 'new_root' and anything mounted below it: these remain mounted
and reachable after the transition and keep being synced normally
as part of their regular life cycle, so they don't need to be
force-flushed here.
- API/pseudo file systems (proc, sysfs, cgroupfs, autofs, ...),
network file systems, and overlayfs (which has no backing store
of its own), as determined by the new fstype_is_worth_syncing()
predicate. There is nothing meaningful to flush on any of these,
and more importantly, opening an untriggered autofs mount point
would needlessly trigger it, and opening a stale network mount
could block for a long time - exactly what we are trying to avoid
on this code path.
- Any flavour of FUSE (plain 'fuse', 'fuseblk', or a
'fuse.<subtype>', e.g. sshfs, rclone, gvfs, ntfs-3g, exfat-fuse,
...), classified via the new fstype_is_fuse() predicate in
src/basic/mountpoint-util.c, plus a few other, non-FUSE guest/host
file sharing file systems with the same "backed by a companion
daemon/hypervisor that could be wedged" risk profile (virtiofs,
vboxsf, vmhgfs). All I/O against any of these, including the
syncfs() we'd otherwise issue, is routed through an arbitrary
userspace daemon (or, for virtiofs/vboxsf/vmhgfs, the host/
hypervisor side), which could hang indefinitely if wedged, dead,
or otherwise unresponsive - there's no timeout on this code path.
'fuseblk' might sound exempt given the name, and does wrap an
actual block device, but that doesn't bound its syncfs() latency
by the kernel block layer alone the way a native block device
file system's is: the request is still serviced by the same FUSE
daemon as any other FUSE variant, and can hang exactly the same
way, so it is excluded here too, trading its comparatively minor
data-safety benefit for avoiding that unbounded hang risk.
'9p' (which can be used with a writeback cache and hence carry
real dirty data, e.g. common in QEMU/KVM guests) and the
shared-storage cluster file systems 'gfs', 'gfs2' and 'ocfs2'
(which fstype_is_network() also happens to classify as "network"
file systems, since they additionally rely on a networked
distributed lock manager for coordination) are deliberately *not*
excluded: unlike FUSE/virtiofs/etc., these are serviced by a
mature, in-kernel client (talking directly to the hypervisor over
a bounded virtio transport, or to real - if shared - block
storage), not an arbitrary, potentially wedged userspace daemon,
so they carry the same bounded, local sync latency any other
block device backed file system already does here. Skipping them
would needlessly sacrifice the data-safety guarantee the original
blanket sync() gave them, without meaningfully improving safety.
- Mount table entries that we can positively confirm are currently
shadowed by another mount stacked on top of them at the same
path: since we can only reach a file system by (re-)opening its
target path, and that always resolves to whatever is currently on
top, syncing by path alone could end up flushing the wrong
superblock. Detect this via the new shared
libmount_fs_id_matches_path() helper (factored out of, and now
also used by, the pre-existing get_sub_mounts(), which needed the
exact same check for the same reason). This same check is also
applied to a mountinfo entry whose target is 'new_root' itself
(not just anything strictly below it): comparing its mount ID
against new_root's own, freshly determined mount ID tells apart
the file system that is actually still reachable there (which we
continue to skip) from a stale entry that merely shares the exact
same path (e.g. if new_root wasn't already its own mount point
and got bind-mounted onto itself earlier in switch_root()), which
is departing just the same and must not be skipped just because
of that coincidence.
Every failure mode that means we can no longer be sure we've covered
every departing file system correctly - libmount being unavailable,
/proc/self/mountinfo (or a specific entry in it) failing to parse,
being unable to tell whether a specific entry is currently shadowed,
or syncfs_path() itself failing for an otherwise-eligible entry - is
handled the exact same way: propagate the error up and let the sole
caller, switch_root(), fall back to one plain, global sync() to cover
everything, rather than deciding on and performing that fallback (or,
worse, silently skipping the affected file system without any
fallback at all) at each of these different spots individually. This
should be rare in practice, so it doesn't meaningfully undercut the
benefit of the targeted sync in the common case.
Everything else that's actually about to become unreachable (the old
root itself, but also any other, unrelated real file system that
happens to be mounted underneath it and gets detached along with it)
is still synced, so this keeps the same safety guarantee the original
blanket sync() gave for file systems that actually do go away here.
Uses the existing syncfs_path() helper for the actual open+syncfs.
sync_departing_file_systems() itself returns -EOPNOTSUPP if libmount
support isn't compiled in, handled the same way by switch_root() as
any of its other error returns.
Note we intentionally don't use O_PATH file descriptors here: syncfs()
requires a 'real' file descriptor and fails with EBADF on O_PATH ones.
Also note there remains an inherent, narrow TOCTOU race between the
mount-ID check described above and the open() syncfs_path() performs
right after it: if something else mounts something new on top of a
given 'path' in between, that open() could still end up triggering an
automount, or hanging on a stale mount, since there is no open()/
openat() equivalent of statx()'s AT_NO_AUTOMOUNT to prevent this for a
"real" (non-O_PATH) file descriptor. Unlike the other failure modes
handled here, a hanging open() can't be recovered from by falling back
to sync() afterwards, since control never returns to do so. Closing
this fully would require disproportionate effort (e.g. performing the
open() in a separate, killable/timeout-bounded process) for a window
that is already narrow, since this code only runs with most other
activity on the system already quiesced during the switch_root()
transition itself, so it is accepted as-is (see the comment at the
call site for details).
This mirrors the same reasoning already applied to the shutdown path
in src/shutdown/shutdown.c, which deliberately avoids a 'dumb' sync()
there for identical reasons.
Definitions enumerated under --root= are already rooted. Passing those
paths to the config parsers with the same root prefixes the root again,
so feature and transfer files are parsed from the wrong path.
Repro: create root/etc/sysupdate.d/rootfeat.feature and
01-root.transfer, then run:
build/systemd-sysupdate --root="$root" --verify=no --offline features rootfeat
Before: parsing failed at line 1 with a bogus Source Type= error.
Fixes#42783.
Follow-up for: e1384cfb09
The scan pointer was advanced with utf8_next_char(), which blindly skips
utf8_skip_data[lead byte] bytes, so a line ending in a truncated multibyte
sequence like "foo\xF0" stepped over the NUL and the loop read past the end
of the string. Decode each character with utf8_encoded_to_unichar() and
propagate the error instead of measuring broken characters.
Fixes#43052.
A `\u` escape in `cunescape_one()` was accepted for any 16-bit value
except NUL. That range `0x0000`-`0xFFFF` includes the UTF-16 surrogates
`0xD800`-`0xDFFF`, which are not valid Unicode scalar values and cannot
be encoded as valid UTF-8.
Nowadays, varlink is used to control systemd-udevd. Let's drop the
legacy socket.
Note, the existence of /run/udev/control socket is widely used in both
our code and external projects. Also, the dependency to
systemd-udevd-control.socket is widely used in many projects. Hence, we
need to create a symlink to the socket file and .socket unit file.
Since all executables now manage their required dlopen notes directly
within their own source code with explicit priority levels, it is no
longer necessary to declare high-priority dlopen notes in the shared
libraries themselves.
udev_replace_whitespace() is documented to read at most 'len' bytes from
'str':
- the strspn() skip of leading whitespace stops at a non-space byte or
NUL, not at 'len'
- ata_id passes the space padded, non-NUL-terminated ATA IDENTIFY
model/serial/fw fields
- an all-blank field reads off the end of the 512-byte hd_driveid stack
struct
Capped the skip to 'len'; existing outputs are unchanged. Added a
regression test.
Like strspn(), but reads at most 'n' bytes from the input. strspn()
is bounded only by a non-matching byte or a NUL, so it over-reads a
buffer that is all matching bytes and not NUL terminated within 'n'.
systemd-repart currently forces newly created image files into NOCOW mode.
That prevents files from being reflinked into the image, making image builds
slower and increasing their disk usage on filesystems that support cloning.
Add a tristate --cow= option. By default, leave the filesystem or parent
directory COW policy unchanged. With --cow=yes, explicitly enable COW; with
--cow=no, retain the previous behavior of forcing NOCOW. Add XO_COW as the
counterpart to XO_NOCOW so xopenat_full() applies either policy while retaining
its normal creation-error cleanup.
Document the new option and extend TEST-58-REPART to verify inherited COW and
NOCOW policies as well as explicit COW and NOCOW overrides. Compare the unset
behavior with the filesystem default so the test also works on nodatacow
mounts, and skip it when the inode attribute is unsupported.
Signed-off-by: Daan De Meyer <daan@amutable.com>
Let's take inspiration from string_is_safe() and take a flags field that
allows fine tuning the validation.
Then port over all current users of either function to the new logic.
Note that this *does* change behaviour in various cases:
1. Generally: we'll now always accept the full UAPI.10 alphabet,
including the "~" and "^" characters. As far as I can see there's no
downside to this liberalization as none of the current consumers of
the two functions uses these characters for anything else.
2. systemd-analyze compare-version will now accept version strings with
"_" and "+" without complaining. I see no downside here, it just
normalizes these debugging tools, to make them accept what most our
other tools accept.
3. "bootctl link" will not accept empty version strings anymore
Which is a bugfix I guess.
4. vpick will now refuse "_" and "+" in version strings. It kinda
already did, because when parsing versions from filenames it uses "_"
and "+" as name, architecture and attempt counter separators. We now
systematically refuse it everywhere else in vpick too. This is hence
a clean-up.
Fixes: #28906
Replaces: #42815#41937
Let's bring the version string character set into a systematic order,
matching the order in which they appear and are defined in the UAPI.10
specification text.
This makes it easier to compare the relevant functions.
Allowing this apparently has been cargo-culted from my initial sysupdate
PR, but Claude and me could not find a single other software package
that uses "," as a character within version strings. Hence, let's remove
this, even though this is a compat breakage of a kind, in the hope
nobody notices. We can easily restore this if this later shows to be an
issue for people.
Use checked arithmetic before clamping the journal iovec length.
Stop copying input iovecs once the fixed journal vector is full.
Follow-up for e69cc9eb36
Validate UTF-8 characters against the caller-provided byte slice.
This stops truncated sequences from borrowing bytes past old_length.
Follow-up for e69cc9eb36
Let's the common function for querying the local thread name.
This changes the escaping rules when the therad name is not quite
kosher: previously we'd just escape invalid UTF-8 charcaters, now we do
what we usually do: also escape control characters and such, and limit
us to ASCII.
The description generated here is mostly for debug purposes, and process
names should normally not require this escaping anyway (it's mostly
paranoia), hence I think this change in behaviour should be fine, it's
not part of the API in any form.
efi_get_variable() allocates one byte for probing whether efivarfs
has grown since fstat(), then three more bytes for NUL termination.
Account for both sizes separately so a full readv() result is treated
as concurrent growth and retried before the terminators are written.
Follow-up for ab69a04600
This PR only sets the default to "relaxed" - I can change the default
to "tofu" if desired. But for that we will also need to update the NEWS
file to ensure everyone is aware of this new default.
---
This PR adds a new `systemd.credentials-boot=` kernel
commandline that allows to control if credentials with
a `null` key are accepted.
The possible options are:
* strict: always insist on tpm encryption
* tofu: allow null encryption in firstboot mode and when no tpm is
available
* relaxed: allow null encryption when sb is off, or no tpm is available
* off: allow null encryption always
The default is `relaxed` which is exactly the behavior we had before.
This replaces the initial idea of using plaintext credentials
at firstboot (thanks to Lennart for this nicer and simpler design).
---
With that we can drop `- firstboot: optionally accept credentials at
firstboot without authentication` from TODO.md
Extend real_has_cpu_with_flag to support aarch64 CPU Features using hwcaps.
With this patch, users can find out if their Arm system supports
architecture-specific features such as BTI as follows:
$ systemd-analyze condition 'ConditionCPUFeature=bti'
Signed-off-by: Emanuele Rocca <emanuele.rocca@arm.com>
Previously, even though sd-journal does not support bzip2 and gzip
compression, the dlopen ELF notes for those libraries were still being
attached to libsystemd.so and various journal-handling executables.
This occurred because the unified compression/decompression interfaces
handled all formats unconditionally, causing the compiler and linker to
pull in all associated dlopen notes across the board.
To resolve this, split these functions into generic and journal-specific
variants (e.g., introducing compress_blob_journal() and decompress_blob_journal()).
The journal-specific variants only handle formats actually supported by
the journal (LZ4, XZ, and ZSTD).
By updating sd-journal, journald, journalctl, and related utilities to
use these new `_journal` interfaces and switching them to the narrower
COMPRESS_JOURNAL_NOTE macro, we ensure that unnecessary dlopen notes
(for bzip2 and gzip) are no longer embedded into these binaries.
When compiler optimization or LTO (Link Time Optimization) is enabled,
dlopen helper functions (such as dlopen_libfoo()) can be aggressively
inlined into their callers or cloned for specific call sites.
While this is beneficial for production builds, it alters or completely
removes the original function symbols from the final executable.
Consequently, this breaks the test (to be added in this series) that
checks whether the dlopen_libfoo() function corresponding to a dlopen
ELF note is actually called.
To resolve this, introduce the `_dlopen_loader_` macro. Under developer
builds, it applies the `_noclone_` and `_noinline_` attributes to guarantee
that helper symbols remain intact and discoverable by the test suite. In
production builds, the macro evaluates to nothing, allowing full compiler
optimization to proceed unimpeded.
This also switches all these notes to be defined via SD_ELF_NOTE_DLOPEN_ANCHORED().
Consequently, any notes added within unreachable or unused functions will be
automatically garbage-collected by the linker (--gc-sections) instead of
bloating the final binary.
E.g. unused p11-kit library dependency is now dropped from
systemd-repart.standalone binary.
Before:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
idn Support for internationalized domain names libidn2.so.0 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
p11-kit Support for PKCS11 hardware tokens libp11-kit.so.0 suggested
```
After:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
```
This commit tweaks the in_first_boot() code to make it
a proper shared helper. It tweaks the comments about the
caching policy. This is important because while this is
simialr to `in_initrd()` there is a suble difference:
we never cache the result of access() because other parts
of systemd will use this file to signal if anything about
firstboot changes.
This also tweaks it to return an int instead of a bool.
So any error is returned as -errno. This allows other
parts of systemd (like the random-seed) that need to make
decisions about "unknown" first-boot state.
This also means that the consumers need to get updated
and we can now also use the new helper in random-seed-tool.c
and have only a single helper to check for first-boot.
This commit adds a new `systemd.credentials_boot_policy=` kernel
commandline that allows to control if credentials with
a `null` key are accepted.
The possible options are:
* strict: always insist on tpm encryption
* tofu: allow null encryption in firstboot mode and when no tpm is available
* relaxed: allow null encryption when sb is off, or no tpm is available
* off: allow null encryption always
The default is currently `relaxed` which is the same behavior
as before.
This replaces the initial idea of using plaintext credentials
at firstboot (thanks to Lennart for this nicer and simpler design).
Note that this also moves `in_first_boot()` to `basic/initrd-util`
which is a better fit now.
Don't test xattr support on socket created in filesystem. This doesn't
work in initramfs when /tmp doesn't have tmpfs mounted inside as
initramfs doesn't have xattr support. Setting socket xattr falls back to
parent directory xattr handlers when we work with FS based socket.
Let's test sockfs based socket instead so that the check is generic and
works in all environments.
When a rate limit is armed, for example via StartLimitInterval=, and the
machine is suspended, the rate limit's "clock" is suspended as well,
since the elapse checks use CLOCK_MONOTONIC. This then causes unexpected
situations where a rate limit armed via StartLimitInterval=1h, followed
by a 10 hour suspend, would elapse after 11 hours total.
Let's avoid this by switching the rate limits to CLOCK_BOOTTIME, which
works the same as CLOCK_MONOTONIC, but accounts for the time spent in
suspend as well.
There's one slight concern when it comes to upgrade path - the old
"begin" value of the rate limit is stored as CLOCK_MONOTONIC, but after
upgrading systemd and serializing/deserializing the state it will be
suddenly compared against now(CLOCK_BOOTTIME), which might cause some
rate limits to elapse "prematurely". But this is just a one-time thing,
after which the rate limit timers should re-assess themselves.
Resolves: #42912
Sets store their element in the key slot but use a value destructor
(DEFINE_HASH_OPS_WITH_VALUE_DESTRUCTOR). On the reject/duplicate path
set_ensure_consume() only checked free_key and otherwise called free().
Follow-up for fcc1d0315d
This makes each `foo_cflags` dependency define a `SYSTEMD_CFLAGS_MARKER_FOO`
macro, and checks if the macro is set when headers provided by external
libraries are included.
With this, we can fail fast at compile time if necessary `_cflags`
dependencies are omitted in meson.build. Missing dependencies found by this
mechanism have been added across the tree.
When a fixed date (e.g. 2027-01-01) is paired with a weekday constraint
(e.g. Thu) that does not match, the timer silently never elapses.
Add calendar_spec_from_string_full(..., warn_on_weekday_mismatch) so
user-facing tools can opt in to a log_warning() at parse time:
- systemd-analyze calendar: uses _full(true)
- systemd-run --on-calendar: uses _full(true)
- .timer OnCalendar=: uses log_syntax() with file/line context
Add test_calendar_spec_weekday_conflict(): forks a child with stderr
captured in a memfd via pidref_safe_fork_full(), verifies the warning
is emitted for conflicting specs and suppressed for valid ones.
Fixes: #40350
Signed-off-by: dongshengyuan <dongshengyuan@uniontech.com>
systemd tries to start ctrl-alt-del.target after receiving a request
to reboot in a container (or when ctrl-alt-del is pressed on a real
machine). If the alias is missing, this will fail. Let's provide a
fallback for this too. I don't think it makes sense to make it
configurable: it seems almost everybody is fine with the default
mapping. If not, people can always just create the symlink in the
filesystem.
When we try to build with -static, the linker refuses dlopen and
various functions related to nss. This commit adds the basic support
to compile with -static and allow the link to pass.
Frantisek Sumsal
Eric Curtin
Mike Yuan
dongshengyuan
Armaan Sandhu
Yu Watanabe
Armaan Sandhu
Luca Boccassi
Syed Mohammed Nayyar
Daan De Meyer
Lennart Poettering
Emanuele Rocca
Michael Vogt
Michal Sekletar
Zbigniew Jędrzejewski-Szmek