udev_replace_whitespace() is documented to read at most 'len' bytes from
'str':
- the strspn() skip of leading whitespace stops at a non-space byte or
NUL, not at 'len'
- ata_id passes the space padded, non-NUL-terminated ATA IDENTIFY
model/serial/fw fields
- an all-blank field reads off the end of the 512-byte hd_driveid stack
struct
Capped the skip to 'len'; existing outputs are unchanged. Added a
regression test.
Previously, files listed in 'sources' were built twice:
once when building the main binary, and again when building the
statically linked one.
This change ensures that all object files from the main binary are
reused when building the static binary. Hence, the only step now
necessary for the static binary is linking the object files.
Follow-up for 39d00e1d20.
This PR optimizes the handling of `dlopen` ELF notes and reduces binary
footprints across the tree.
By enabling compiler section-splitting
(`-ffunction-sections`/`-fdata-sections`), the linker can now accurately
garbage-collect unused code and data. To align with this,
`SD_ELF_NOTE_DLOPEN_ANCHORED()` macro is introduced, which ties `dlopen`
notes to their calling functions so that unused notes are automatically
removed by `--gc-sections`.
Additionally, this explicitly embeds required `dlopen` notes into
individual executables to fix a visibility issue where package managers
missed runtime dependencies invoked indirectly through
`libsystemd-shared.so`.
Even though systemd-networkd and systemd-udevd do not support
ConditionSecurity= in .network, .netdev, and .link files, the unified
condition_test() function maintained a function pointer table that
unconditionally referenced condition_test_security(). Consequently,
linker garbage collection (--gc-sections) could not drop the security
test code, inadvertently pulling in dlopen dependencies and ELF notes
for apparmor, audit, and tpm2 libraries into these binaries.
To resolve this, introduce condition_test_net() and condition_test_list_net(),
which utilize a trimmed-down function pointer table that excludes
security-related condition evaluators.
By migrating networkd and udevd to these new network-specific variants,
the reference to condition_test_security() is completely severed in
their dependency chains. This allows the compiler and linker to
successfully garbage-collect the unused security logic and safely drops
the unnecessary dlopen notes from those binaries.
Previously, even though sd-journal does not support bzip2 and gzip
compression, the dlopen ELF notes for those libraries were still being
attached to libsystemd.so and various journal-handling executables.
This occurred because the unified compression/decompression interfaces
handled all formats unconditionally, causing the compiler and linker to
pull in all associated dlopen notes across the board.
To resolve this, split these functions into generic and journal-specific
variants (e.g., introducing compress_blob_journal() and decompress_blob_journal()).
The journal-specific variants only handle formats actually supported by
the journal (LZ4, XZ, and ZSTD).
By updating sd-journal, journald, journalctl, and related utilities to
use these new `_journal` interfaces and switching them to the narrower
COMPRESS_JOURNAL_NOTE macro, we ensure that unnecessary dlopen notes
(for bzip2 and gzip) are no longer embedded into these binaries.
Enable the `dlopen` unit test suite in GitHub Actions, except for the
following configurations where linker garbage collection (`--gc-sections`)
or dead-code elimination fails to drop unused dlopen symbols:
- ppc64le (both GCC and Clang)
- s390x (GCC only)
- Sanitizer-enabled setups
This introduces a test utility to validate the presence and correctness
of dlopen ELF notes.
The test extracts and compares dlopen ELF notes between the dynamic and
static versions of target binaries to catch missing entries. Furthermore,
it scans the code for dlopen_foo() invocations to ensure that no stale or
unused notes remain in the final binary, thereby verifying the linker's
garbage-collection integration.
When compiler optimization or LTO (Link Time Optimization) is enabled,
dlopen helper functions (such as dlopen_libfoo()) can be aggressively
inlined into their callers or cloned for specific call sites.
While this is beneficial for production builds, it alters or completely
removes the original function symbols from the final executable.
Consequently, this breaks the test (to be added in this series) that
checks whether the dlopen_libfoo() function corresponding to a dlopen
ELF note is actually called.
To resolve this, introduce the `_dlopen_loader_` macro. Under developer
builds, it applies the `_noclone_` and `_noinline_` attributes to guarantee
that helper symbols remain intact and discoverable by the test suite. In
production builds, the macro evaluates to nothing, allowing full compiler
optimization to proceed unimpeded.
Previously, when a library was dynamically loaded via a helper function
inside libsystemd-shared.so, the resulting dlopen ELF note was not
propagated to the invoking executable's ELF metadata.
This commit explicitly annotates each executable with the relevant
dlopen ELF notes for any optional dependencies it might potentially
load. This ensures that package managers and build systems can properly
discover runtime dependencies that are triggered indirectly.
The new SD_ELF_NOTE_DLOPEN_ANCHORED() macro utilizes the 'o' assembler
flag (SHF_LINK_ORDER), which requires binutils >= 2.35 or LLVM >= 18.
If an LLVM version older than 18 is encountered, the macro automatically
falls back to the non-anchored variant.
This non-anchored fallback relies on the 'R' (SHF_GNU_RETAIN) flag,
which requires binutils >= 2.36 or LLVM >= 13.
Since the codebase now unconditionally adopts SD_ELF_NOTE_DLOPEN_ANCHORED()
tree-wide, the effective minimum toolchain requirements become:
- binutils >= 2.35 (to support the 'o' flag)
- LLVM/Clang >= 13 (to support the 'R' flag fallback for versions < 18)
Update the minimal toolchain versions in the README to reflect these
requirements for building systemd.
This also switches all these notes to be defined via SD_ELF_NOTE_DLOPEN_ANCHORED().
Consequently, any notes added within unreachable or unused functions will be
automatically garbage-collected by the linker (--gc-sections) instead of
bloating the final binary.
E.g. unused p11-kit library dependency is now dropped from
systemd-repart.standalone binary.
Before:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
idn Support for internationalized domain names libidn2.so.0 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
p11-kit Support for PKCS11 hardware tokens libp11-kit.so.0 suggested
```
After:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
```
SD_ELF_NOTE_DLOPEN_ANCHORED() emits a .note.dlopen ELF note that is
"anchored" to a dummy symbol via the SHF_LINK_ORDER ('o') section flag,
in addition to SHF_GROUP ('G') for folding identical notes together.
Unlike the plain SD_ELF_NOTE_DLOPEN() macro, this variant ties the
note's lifetime to a dummy symbol named with the specified tag: if the
linker's --gc-sections removes the function that calls the macro (e.g.
because the function is never referenced), the associated .note.dlopen
entry is garbage-collected along with it.
Currently, the new macro is unused in our code, but all dlopen notes
will be generated with this in later commits.
Extend the module building logic to automatically generate '.standalone'
variants for non-NSS/PAM shared modules (such as cryptsetup tokens).
Like standalone executables, these are not built by default to keep
the base build time unaffected, but can be built explicitly on demand
via `ninja <module_name>.standalone`.
Extend the 'install' keyword for executable definitions to accept four
modes—'yes', 'no', 'both', and 'static'—to elegantly manage the creation
and installation of both shared and statically-linked (.standalone) binaries.
- 'yes' / 'no': Standard behavior (mapped to true/false).
- 'both': Installs both the shared and static variants.
- 'static': Installs the static variant under the original name, while
suffixing the uninstalled shared variant with '.shared'.
With this change, any arbitrary executable can now have its `.standalone`
variant built on demand simply by invoking `ninja <target>.standalone`.
For example, `varlinkctl` did not previously support a standalone variant,
but it can now be built explicitly via `ninja varlinkctl.standalone`.
These `.standalone` binaries are not built by default unless explicitly
specified as a ninja target or enabled via `-Dstandalone-binaries=true`.
Thus, the default build time should remain unaffected.
This centralisation eliminates a massive amount of boilerplate and duplicated
target declarations across almost all subdirectories (e.g., systemd-repart,
systemd-tmpfiles, systemd-shutdown, and systemd-report tools).
systemd-resolved already has a dns_scope_free() function in
resolved-dns-scope.c for DnsScope. Since the one in dns-configuration.c
is only used internally, make it static.
This is necessary to allow systemd-resolved to be statically linked
with libsystemd-shared without symbol conflicts.
This makes each `foo_cflags` dependency define a `SYSTEMD_CFLAGS_MARKER_FOO`
macro, and checks if the macro is set when headers provided by external
libraries are included.
With this, we can fail fast at compile time if necessary `_cflags`
dependencies are omitted in meson.build. Missing dependencies found by this
mechanism have been added across the tree.
Overriding <linux/audit.h> to implicitly include <libaudit.h> turns out to
be problematic. Because <linux/audit.h> is pulled in by various other kernel
headers, any source file including those kernel headers indirectly ends up
depending on <libaudit.h>, even if the executable or library being built
does not require libaudit at all.
Let's drop the override header and include <libaudit.h> only where explicitly
needed. This also moves the fallback definitions of AUDIT_SERVICE_* and
MAX_AUDIT_MESSAGE_LENGTH to libaudit-util.h.
Obserbed with GCC-11 on Ubuntu.
```
In file included from ../src/shared/format-table.h:7,
from ../src/bless-boot/bless-boot.c:11:
../src/bless-boot/bless-boot.c: In function ‘verb_set’:
../src/basic/log.h:187:27: error: ‘source2’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
187 | ? log_internal(_level, _e, PROJECT_FILE, __LINE__, __func__, __VA_ARGS__) \
| ^~~~~~~~~~~~
../src/bless-boot/bless-boot.c:458:40: note: ‘source2’ was declared here
458 | const char *target, *source1, *source2;
| ^~~~~~~
In file included from ../src/shared/format-table.h:7,
from ../src/bless-boot/bless-boot.c:11:
../src/basic/log.h:187:27: error: ‘source1’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
187 | ? log_internal(_level, _e, PROJECT_FILE, __LINE__, __func__, __VA_ARGS__) \
| ^~~~~~~~~~~~
../src/bless-boot/bless-boot.c:458:30: note: ‘source1’ was declared here
458 | const char *target, *source1, *source2;
| ^~~~~~~
In file included from ../src/shared/format-table.h:7,
from ../src/bless-boot/bless-boot.c:11:
../src/basic/log.h:187:27: error: ‘target’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
187 | ? log_internal(_level, _e, PROJECT_FILE, __LINE__, __func__, __VA_ARGS__) \
| ^~~~~~~~~~~~
../src/bless-boot/bless-boot.c:458:21: note: ‘target’ was declared here
458 | const char *target, *source1, *source2;
| ^~~~~~
cc1: all warnings being treated as errors
```
Previously, if systemd was built with OpenSSL UI support, it would fail
to load libcrypto at runtime if the library lacked UI support, requiring
a recompilation of systemd to fix.
Let's relax this strict requirement by making the UI methods optional
during dlopen(). openssl_ui_supported() is added to dynamically check
if all required UI symbols were successfully loaded.
OpenSSL UI is not a mandatory feature to load private keys from an engine
or a provider. Let's allow loading private keys even if OpenSSL UI is not
supported.
Note that even if OPENSSL_NO_UI_CONSOLE is set, the type UI_METHOD is
always defined. Hence, the `#ifndef` condition in the definition of
struct OpenSSLAskPasswordUI is unnecessary and can be dropped.
If systemd is compiled with OpenSSL 3 headers but executed in an environment
where OpenSSL 4 (libcrypto.so.4) is loaded, dlopen_many_sym_or_warn() will
fail because OpenSSL 4 completely removes the deprecated ENGINE API. This
breaks the ability to dynamically fallback and seamlessly upgrade OpenSSL
without recompiling systemd.
To fix this, drop the ENGINE API symbols from the mandatory DLSYM_ARG() list.
Instead, try to load them via DLSYM_OPTIONAL() after the library is opened.
load_key_from_engine() is updated to check for their presence and return
-EOPNOTSUPP if the loaded OpenSSL version does not provide them.
Currently, rsa_pkey_from_n_e() uses architecture-specific `#if` branches
and memdup_reverse() to handle big-endian RSA components (n and e)
before passing them directly to OSSL_PARAM_construct_BN().
We can simplify this by parsing the raw big-endian bytes into BIGNUMs
first using BN_bin2bn(), which natively expects big-endian data. We
can then push these BIGNUMs into OSSL_PARAM_BLD. This delegates the
data format handling entirely to OpenSSL and successfully removes the
platform-specific code.
Drop memstream and i2d_PUBKEY_fp(). We can simply use i2d_PUBKEY()
which automatically allocates the necessary buffer for us.
Note that dropping the secure erase (erase_and_freep()) in favor of
OPENSSL_free() is intentional and safe, as the buffer only holds
public key material which does not need to be securely wiped.
Buffers allocated by OpenSSL must be freed with OPENSSL_free().
Fortunately, we do not enable the secure heap, so OPENSSL_free()
is currently equivalent to free(), but let's fix this for correctness.
OpenSSL 3.0 deprecated low-level Elliptic Curve (EC) key manipulation
functions (EC_KEY, EC_POINT, EC_GROUP) and direct signature verification
functions like ECDSA_do_verify().
This commit modernizes dnssec_ecdsa_verify_raw() by transitioning to the
provider-aware EVP API:
* Uses OSSL_PARAM arrays and EVP_PKEY_fromdata() to construct the EC
public key directly from the raw octet string and curve name, avoiding
deprecated EC_POINT parsing.
* Converts the raw R and S signature components into a DER-encoded ASN.1
signature using i2d_ECDSA_SIG(), as required by the modern EVP API.
* Uses EVP_PKEY_verify() for the actual signature validation.
Additionally, this drops an outdated TODO comment waiting for raw ECDSA
support in the EVP API, as well as the deprecated warning suppression
macros and fallback code blocks. Unit tests for ECDSA are now run
unconditionally.
OpenSSL 3.0 deprecated low-level key manipulation functions and direct
access to RSA structures (such as RSA_new(), RSA_set0_key(), and
RSA_size()).
This commit modernizes dnssec_rsa_verify_raw() by replacing these
deprecated functions with the provider-aware EVP API:
* Uses OSSL_PARAM_BLD and EVP_PKEY_fromdata() to construct the RSA
public key directly from the modulus and exponent BIGNUMs.
* Replaces RSA_size() with EVP_PKEY_get_size().
Consequently, the workaround macros suppressing deprecated warnings
(DISABLE_WARNING_DEPRECATED_DECLARATIONS) and the conditional fallback
blocks (#if !defined(OPENSSL_NO_DEPRECATED_3_0)) are no longer needed
and have been dropped. Unit tests are also updated to run unconditionally.
They will be used in later commits.
Note, ECDSA_SIG_new() and ECDSA_SIG_set0() are not deprecated.
They are moved to the section for non-deprecated symbols.
Both glibc and musl provides dgettext(). Hence it is not necessary to
use libintl.so provided by gettext.
This partially reverts 590e226437,
fully reverts commit e6e65dc261 and
bd19ffd9cb.
Then, introduce minimal libintl.h for musl build, to avoid using
libintl.h by gettext, which is typically installed on musl-based
build systems.
This reverts commit 7f33ee8bb4.
The file is located outside mkosi/ subdirectory, hence currently unused.
If this is moved to mkosi/ subdirectory, the config conflicts with
TEST-58-REPART. Let's remove it at least now, and reintroduce it later
at correct place with test adjustment if this is really useful.
With this change, gcrypt dependency is not mandatory. Hence, allow to build
systemd even when -D gcrypt=enabled but gcrypt devel package is not installed.
This introduce a vtable for journal tagging feature in sd-journal,
and makes libgcrypt dependent features loaded by users (journald,
journalctl, journal-remote, and unit tests) when necessary.
Then, move several components for journal tagging in JournalFile
to JournalAuthContext.
This also introduces wrapper functions that checks gcrypt support.
Observerd on Ubuntu 24.04 with GCC 13 on arm64 architecture.
```
../src/shared/journal-importer.c: In function ‘journal_importer_process_data’:
../src/shared/journal-importer.c:344:30: error: ‘line’ may be used uninitialized [-Werror=maybe-uninitialized]
344 | if (!journal_field_valid(line, n - 1, true)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/shared/journal-importer.c:295:23: note: ‘line’ was declared here
295 | char *line, *sep;
| ^~~~
cc1: all warnings being treated as errors
```
There are already varlink methods for `ResolveHostname`,
`ResolveAddress`, and `ResolveRecord`. Use those in `resolvectl` instead
of the dbus equivalents.
postmarketOS image can be built with for example the following mkosi.local.conf:
```
[Distribution]
Distribution=postmarketos
[Output]
Format=disk
[Build]
UseSubvolumes=yes
[Runtime]
Firmware=uefi
```
In Meta production we have been considering using journald more widely
for some time. One of the blockers to doing that which I have noticed is
that often journald seems to have vastly less data after lockups/power
failures compared to plain files, which is not great when debugging
outages.
On small write rates this tends to be hard to reproduce, but when
writing thousands of messages a second, an unclean shutdown can result
in the end result being an active journal file with a header that
records an arena larger than the data that actually reached disk. What
happens is then that journalctl then discards the entire file(!),
completely ignoring that there is a huge amount of data which is
actually perfectly readable.
The reason for that is that the journal header is updated on every
append, while the file size and newly written arena contents are only
made durable on the filesystem's own schedule. After a crash, the header
can therefore describe writes which were logically completed by journald
but whose backing data or file metadata never reached disk.
Take the following example of how this can happen at high log rates:
1. journald appends objects into an mmap()ed arena, periodically growing
the file with fallocate() in FILE_SIZE_INCREASE (8M) steps and advancing
the header's arena_size tail pointers as it goes along.
2. The header is dirtied on every append, and its arena_size is advanced
at each fallocate(). It is, from the kernel's perspective, an ordinary
data page and is only made durable by the kernel's periodic page cache
writeback on its own schedule. The file's length, by contrast, is
metadata, made durable only when the filesystem commits a transaction
(or on an fsync(), which journald does not issue between sync
intervals).
3. journald marks journals NOCOW, so the header's data block is
overwritten in place and is decoupled from the size metadata. Nothing
orders the two with respect to each other. Writeback therefore can
routinely persist a header whose arena_size has run ahead of the file
length recorded on disk.
4. Power is lost. On the next boot the persisted header reflects an
arena_size and tail pointers which have been advanced for appends.
However their payload and the file metadata were never committed, so
header_size + arena_size now points well past the end of the file as it
exists on disk.
5. journal_file_verify_header() then rejects this with -ENODATA:
if (... || header_size + arena_size > (uint64_t) f->last_stat.st_size)
return -ENODATA;
That is correct when opening for writing, because we must not append to
a file whose recorded state we cannot trust, and the caller must rotate
it away. But the same check also runs on read only opens, where it is
actively harmful. In the case of journalctl, the entire file is skipped,
even though the data hash table, the field hash table, and the head of
the array all are present and fully intact, and the great majority of
entries are physically present. In fact, only a very small part of the
most recently written tail is missing, but everything before is
readable. This results in mistakenly rejecting the entire file as
corrupt.
This happens extremely frequently on machines with high write rates
during power cuts or lockups. In testing writing ~7500 msg/s through
journald and then cutting power, I reproduced it in ten out of ten
attempts across different machines.
In each case, the header was left claiming ~296M of arena while only
~192-208M had reached disk. In this case, journalctl reports that it has
recovered 0 of ~335000 messages. Whether a given crash trips the
condition depends on where it falls relative to the header's writeback,
but when it does, the loss today is total. After this patch the vast
majority of messages can be retrieved.
Let's fix this by keeping the rejection for writing, but for read-only
opens, let's just clamp the arena to the real file size and skip the
consistency checks on the now unreliable tail pointers. The reader will
walk the entry array chain from its intact head and stop at the
truncation point by the bounds check that already exists, so there's no
need to do any more than that there.
On i386 with musl and libucontext, many tests crash with SIGSEGV
when using fibers implemented on top of ucontext APIs.
For example:
```
929/1498 libsystemd - systemd:test-event-future FAIL 0.97s killed by signal 11 SIGSEGV
/* test_sd_event_run_timer */
run-suspend-timer: Scheduling fiber
/home/pmos/build/src/systemd-261-rc3/tools/test-crash-trace.sh: line 18: 19994 Segmentation fault (core dumped) "$@"
===== exit 139 — replaying under gdb =====
/* test_sd_event_run_timer */
run-suspend-timer: Scheduling fiber
Program received signal SIGSEGV, Segmentation fault.
0xf7d68be3 in sd_event_add_time_relative (e=0xf7ffd3f0, ret=0xf7b01fa0, clock=1, usec=10000, accuracy=0, callback=0x565564ad <inner_timer_handler>, userdata=0xf7b01fa4) at ../src/libsystemd/sd-event/sd-event.c:1455
1455 void *userdata) {
Thread 1 (process 20485 "test-event-futu"):
#0 0xf7d68be3 in sd_event_add_time_relative (e=0xf7ffd3f0, ret=0xf7b01fa0, clock=1, usec=10000, accuracy=0, callback=0x565564ad <inner_timer_handler>, userdata=0xf7b01fa4) at ../src/libsystemd/sd-event/sd-event.c:1455
t = 17868423377094431728
r = <optimized out>
#1 0x565574c7 in sd_event_run_timer_fiber (userdata=0x0) at ../src/libsystemd/sd-event/test-event-future.c:329
inner = 0xf7ffd3f0
source = 0x0
counter = 0
r = <optimized out>
#2 0xf7dab308 in fiber_entry_point () at ../src/libsystemd/sd-future/fiber.c:208
_cleanup_log_unset_prefix_5 = 0x0
__unique_prefix_c6 = 0x5655bb30
f = 0xf7b06840
__func__ = "fiber_entry_point"
fake_stack_save = <optimized out>
#3 0xf7b032ae in setcontext () from /lib/libucontext.so.1
No symbol table info available.
#4 0x00000000 in ?? ()
No symbol table info available.
```
Building systemd with -mstackrealign makes all observed failures
disappear, suggesting a stack alignment issue in the interaction
between compiler-generated code and the ucontext implementation.
This resembles historical stack alignment issues in glibc's i386
makecontext() implementation, which required fixes to preserve the
expected stack alignment.
As a workaround, add -mstackrealign when building on i386 with musl
and libucontext.
Suggested-by: ChatGPT (OpenAI)
Yu Watanabe