This also switches all these notes to be defined via SD_ELF_NOTE_DLOPEN_ANCHORED().
Consequently, any notes added within unreachable or unused functions will be
automatically garbage-collected by the linker (--gc-sections) instead of
bloating the final binary.
E.g. unused p11-kit library dependency is now dropped from
systemd-repart.standalone binary.
Before:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
idn Support for internationalized domain names libidn2.so.0 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
p11-kit Support for PKCS11 hardware tokens libp11-kit.so.0 suggested
```
After:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
```
SD_ELF_NOTE_DLOPEN_ANCHORED() emits a .note.dlopen ELF note that is
"anchored" to a dummy symbol via the SHF_LINK_ORDER ('o') section flag,
in addition to SHF_GROUP ('G') for folding identical notes together.
Unlike the plain SD_ELF_NOTE_DLOPEN() macro, this variant ties the
note's lifetime to a dummy symbol named with the specified tag: if the
linker's --gc-sections removes the function that calls the macro (e.g.
because the function is never referenced), the associated .note.dlopen
entry is garbage-collected along with it.
Currently, the new macro is unused in our code, but all dlopen notes
will be generated with this in later commits.
Extend the module building logic to automatically generate '.standalone'
variants for non-NSS/PAM shared modules (such as cryptsetup tokens).
Like standalone executables, these are not built by default to keep
the base build time unaffected, but can be built explicitly on demand
via `ninja <module_name>.standalone`.
Extend the 'install' keyword for executable definitions to accept four
modes—'yes', 'no', 'both', and 'static'—to elegantly manage the creation
and installation of both shared and statically-linked (.standalone) binaries.
- 'yes' / 'no': Standard behavior (mapped to true/false).
- 'both': Installs both the shared and static variants.
- 'static': Installs the static variant under the original name, while
suffixing the uninstalled shared variant with '.shared'.
With this change, any arbitrary executable can now have its `.standalone`
variant built on demand simply by invoking `ninja <target>.standalone`.
For example, `varlinkctl` did not previously support a standalone variant,
but it can now be built explicitly via `ninja varlinkctl.standalone`.
These `.standalone` binaries are not built by default unless explicitly
specified as a ninja target or enabled via `-Dstandalone-binaries=true`.
Thus, the default build time should remain unaffected.
This centralisation eliminates a massive amount of boilerplate and duplicated
target declarations across almost all subdirectories (e.g., systemd-repart,
systemd-tmpfiles, systemd-shutdown, and systemd-report tools).
systemd-resolved already has a dns_scope_free() function in
resolved-dns-scope.c for DnsScope. Since the one in dns-configuration.c
is only used internally, make it static.
This is necessary to allow systemd-resolved to be statically linked
with libsystemd-shared without symbol conflicts.
This makes each `foo_cflags` dependency define a `SYSTEMD_CFLAGS_MARKER_FOO`
macro, and checks if the macro is set when headers provided by external
libraries are included.
With this, we can fail fast at compile time if necessary `_cflags`
dependencies are omitted in meson.build. Missing dependencies found by this
mechanism have been added across the tree.
Overriding <linux/audit.h> to implicitly include <libaudit.h> turns out to
be problematic. Because <linux/audit.h> is pulled in by various other kernel
headers, any source file including those kernel headers indirectly ends up
depending on <libaudit.h>, even if the executable or library being built
does not require libaudit at all.
Let's drop the override header and include <libaudit.h> only where explicitly
needed. This also moves the fallback definitions of AUDIT_SERVICE_* and
MAX_AUDIT_MESSAGE_LENGTH to libaudit-util.h.
Obserbed with GCC-11 on Ubuntu.
```
In file included from ../src/shared/format-table.h:7,
from ../src/bless-boot/bless-boot.c:11:
../src/bless-boot/bless-boot.c: In function ‘verb_set’:
../src/basic/log.h:187:27: error: ‘source2’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
187 | ? log_internal(_level, _e, PROJECT_FILE, __LINE__, __func__, __VA_ARGS__) \
| ^~~~~~~~~~~~
../src/bless-boot/bless-boot.c:458:40: note: ‘source2’ was declared here
458 | const char *target, *source1, *source2;
| ^~~~~~~
In file included from ../src/shared/format-table.h:7,
from ../src/bless-boot/bless-boot.c:11:
../src/basic/log.h:187:27: error: ‘source1’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
187 | ? log_internal(_level, _e, PROJECT_FILE, __LINE__, __func__, __VA_ARGS__) \
| ^~~~~~~~~~~~
../src/bless-boot/bless-boot.c:458:30: note: ‘source1’ was declared here
458 | const char *target, *source1, *source2;
| ^~~~~~~
In file included from ../src/shared/format-table.h:7,
from ../src/bless-boot/bless-boot.c:11:
../src/basic/log.h:187:27: error: ‘target’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
187 | ? log_internal(_level, _e, PROJECT_FILE, __LINE__, __func__, __VA_ARGS__) \
| ^~~~~~~~~~~~
../src/bless-boot/bless-boot.c:458:21: note: ‘target’ was declared here
458 | const char *target, *source1, *source2;
| ^~~~~~
cc1: all warnings being treated as errors
```
Previously, if systemd was built with OpenSSL UI support, it would fail
to load libcrypto at runtime if the library lacked UI support, requiring
a recompilation of systemd to fix.
Let's relax this strict requirement by making the UI methods optional
during dlopen(). openssl_ui_supported() is added to dynamically check
if all required UI symbols were successfully loaded.
OpenSSL UI is not a mandatory feature to load private keys from an engine
or a provider. Let's allow loading private keys even if OpenSSL UI is not
supported.
Note that even if OPENSSL_NO_UI_CONSOLE is set, the type UI_METHOD is
always defined. Hence, the `#ifndef` condition in the definition of
struct OpenSSLAskPasswordUI is unnecessary and can be dropped.
If systemd is compiled with OpenSSL 3 headers but executed in an environment
where OpenSSL 4 (libcrypto.so.4) is loaded, dlopen_many_sym_or_warn() will
fail because OpenSSL 4 completely removes the deprecated ENGINE API. This
breaks the ability to dynamically fallback and seamlessly upgrade OpenSSL
without recompiling systemd.
To fix this, drop the ENGINE API symbols from the mandatory DLSYM_ARG() list.
Instead, try to load them via DLSYM_OPTIONAL() after the library is opened.
load_key_from_engine() is updated to check for their presence and return
-EOPNOTSUPP if the loaded OpenSSL version does not provide them.
Currently, rsa_pkey_from_n_e() uses architecture-specific `#if` branches
and memdup_reverse() to handle big-endian RSA components (n and e)
before passing them directly to OSSL_PARAM_construct_BN().
We can simplify this by parsing the raw big-endian bytes into BIGNUMs
first using BN_bin2bn(), which natively expects big-endian data. We
can then push these BIGNUMs into OSSL_PARAM_BLD. This delegates the
data format handling entirely to OpenSSL and successfully removes the
platform-specific code.
Drop memstream and i2d_PUBKEY_fp(). We can simply use i2d_PUBKEY()
which automatically allocates the necessary buffer for us.
Note that dropping the secure erase (erase_and_freep()) in favor of
OPENSSL_free() is intentional and safe, as the buffer only holds
public key material which does not need to be securely wiped.
Buffers allocated by OpenSSL must be freed with OPENSSL_free().
Fortunately, we do not enable the secure heap, so OPENSSL_free()
is currently equivalent to free(), but let's fix this for correctness.
OpenSSL 3.0 deprecated low-level Elliptic Curve (EC) key manipulation
functions (EC_KEY, EC_POINT, EC_GROUP) and direct signature verification
functions like ECDSA_do_verify().
This commit modernizes dnssec_ecdsa_verify_raw() by transitioning to the
provider-aware EVP API:
* Uses OSSL_PARAM arrays and EVP_PKEY_fromdata() to construct the EC
public key directly from the raw octet string and curve name, avoiding
deprecated EC_POINT parsing.
* Converts the raw R and S signature components into a DER-encoded ASN.1
signature using i2d_ECDSA_SIG(), as required by the modern EVP API.
* Uses EVP_PKEY_verify() for the actual signature validation.
Additionally, this drops an outdated TODO comment waiting for raw ECDSA
support in the EVP API, as well as the deprecated warning suppression
macros and fallback code blocks. Unit tests for ECDSA are now run
unconditionally.
OpenSSL 3.0 deprecated low-level key manipulation functions and direct
access to RSA structures (such as RSA_new(), RSA_set0_key(), and
RSA_size()).
This commit modernizes dnssec_rsa_verify_raw() by replacing these
deprecated functions with the provider-aware EVP API:
* Uses OSSL_PARAM_BLD and EVP_PKEY_fromdata() to construct the RSA
public key directly from the modulus and exponent BIGNUMs.
* Replaces RSA_size() with EVP_PKEY_get_size().
Consequently, the workaround macros suppressing deprecated warnings
(DISABLE_WARNING_DEPRECATED_DECLARATIONS) and the conditional fallback
blocks (#if !defined(OPENSSL_NO_DEPRECATED_3_0)) are no longer needed
and have been dropped. Unit tests are also updated to run unconditionally.
They will be used in later commits.
Note, ECDSA_SIG_new() and ECDSA_SIG_set0() are not deprecated.
They are moved to the section for non-deprecated symbols.
Both glibc and musl provides dgettext(). Hence it is not necessary to
use libintl.so provided by gettext.
This partially reverts 590e226437,
fully reverts commit e6e65dc261 and
bd19ffd9cb.
Then, introduce minimal libintl.h for musl build, to avoid using
libintl.h by gettext, which is typically installed on musl-based
build systems.
This reverts commit 7f33ee8bb4.
The file is located outside mkosi/ subdirectory, hence currently unused.
If this is moved to mkosi/ subdirectory, the config conflicts with
TEST-58-REPART. Let's remove it at least now, and reintroduce it later
at correct place with test adjustment if this is really useful.
With this change, gcrypt dependency is not mandatory. Hence, allow to build
systemd even when -D gcrypt=enabled but gcrypt devel package is not installed.
This introduce a vtable for journal tagging feature in sd-journal,
and makes libgcrypt dependent features loaded by users (journald,
journalctl, journal-remote, and unit tests) when necessary.
Then, move several components for journal tagging in JournalFile
to JournalAuthContext.
This also introduces wrapper functions that checks gcrypt support.
Observerd on Ubuntu 24.04 with GCC 13 on arm64 architecture.
```
../src/shared/journal-importer.c: In function ‘journal_importer_process_data’:
../src/shared/journal-importer.c:344:30: error: ‘line’ may be used uninitialized [-Werror=maybe-uninitialized]
344 | if (!journal_field_valid(line, n - 1, true)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/shared/journal-importer.c:295:23: note: ‘line’ was declared here
295 | char *line, *sep;
| ^~~~
cc1: all warnings being treated as errors
```
There are already varlink methods for `ResolveHostname`,
`ResolveAddress`, and `ResolveRecord`. Use those in `resolvectl` instead
of the dbus equivalents.
postmarketOS image can be built with for example the following mkosi.local.conf:
```
[Distribution]
Distribution=postmarketos
[Output]
Format=disk
[Build]
UseSubvolumes=yes
[Runtime]
Firmware=uefi
```
In Meta production we have been considering using journald more widely
for some time. One of the blockers to doing that which I have noticed is
that often journald seems to have vastly less data after lockups/power
failures compared to plain files, which is not great when debugging
outages.
On small write rates this tends to be hard to reproduce, but when
writing thousands of messages a second, an unclean shutdown can result
in the end result being an active journal file with a header that
records an arena larger than the data that actually reached disk. What
happens is then that journalctl then discards the entire file(!),
completely ignoring that there is a huge amount of data which is
actually perfectly readable.
The reason for that is that the journal header is updated on every
append, while the file size and newly written arena contents are only
made durable on the filesystem's own schedule. After a crash, the header
can therefore describe writes which were logically completed by journald
but whose backing data or file metadata never reached disk.
Take the following example of how this can happen at high log rates:
1. journald appends objects into an mmap()ed arena, periodically growing
the file with fallocate() in FILE_SIZE_INCREASE (8M) steps and advancing
the header's arena_size tail pointers as it goes along.
2. The header is dirtied on every append, and its arena_size is advanced
at each fallocate(). It is, from the kernel's perspective, an ordinary
data page and is only made durable by the kernel's periodic page cache
writeback on its own schedule. The file's length, by contrast, is
metadata, made durable only when the filesystem commits a transaction
(or on an fsync(), which journald does not issue between sync
intervals).
3. journald marks journals NOCOW, so the header's data block is
overwritten in place and is decoupled from the size metadata. Nothing
orders the two with respect to each other. Writeback therefore can
routinely persist a header whose arena_size has run ahead of the file
length recorded on disk.
4. Power is lost. On the next boot the persisted header reflects an
arena_size and tail pointers which have been advanced for appends.
However their payload and the file metadata were never committed, so
header_size + arena_size now points well past the end of the file as it
exists on disk.
5. journal_file_verify_header() then rejects this with -ENODATA:
if (... || header_size + arena_size > (uint64_t) f->last_stat.st_size)
return -ENODATA;
That is correct when opening for writing, because we must not append to
a file whose recorded state we cannot trust, and the caller must rotate
it away. But the same check also runs on read only opens, where it is
actively harmful. In the case of journalctl, the entire file is skipped,
even though the data hash table, the field hash table, and the head of
the array all are present and fully intact, and the great majority of
entries are physically present. In fact, only a very small part of the
most recently written tail is missing, but everything before is
readable. This results in mistakenly rejecting the entire file as
corrupt.
This happens extremely frequently on machines with high write rates
during power cuts or lockups. In testing writing ~7500 msg/s through
journald and then cutting power, I reproduced it in ten out of ten
attempts across different machines.
In each case, the header was left claiming ~296M of arena while only
~192-208M had reached disk. In this case, journalctl reports that it has
recovered 0 of ~335000 messages. Whether a given crash trips the
condition depends on where it falls relative to the header's writeback,
but when it does, the loss today is total. After this patch the vast
majority of messages can be retrieved.
Let's fix this by keeping the rejection for writing, but for read-only
opens, let's just clamp the arena to the real file size and skip the
consistency checks on the now unreliable tail pointers. The reader will
walk the entry array chain from its intact head and stop at the
truncation point by the bounds check that already exists, so there's no
need to do any more than that there.
On i386 with musl and libucontext, many tests crash with SIGSEGV
when using fibers implemented on top of ucontext APIs.
For example:
```
929/1498 libsystemd - systemd:test-event-future FAIL 0.97s killed by signal 11 SIGSEGV
/* test_sd_event_run_timer */
run-suspend-timer: Scheduling fiber
/home/pmos/build/src/systemd-261-rc3/tools/test-crash-trace.sh: line 18: 19994 Segmentation fault (core dumped) "$@"
===== exit 139 — replaying under gdb =====
/* test_sd_event_run_timer */
run-suspend-timer: Scheduling fiber
Program received signal SIGSEGV, Segmentation fault.
0xf7d68be3 in sd_event_add_time_relative (e=0xf7ffd3f0, ret=0xf7b01fa0, clock=1, usec=10000, accuracy=0, callback=0x565564ad <inner_timer_handler>, userdata=0xf7b01fa4) at ../src/libsystemd/sd-event/sd-event.c:1455
1455 void *userdata) {
Thread 1 (process 20485 "test-event-futu"):
#0 0xf7d68be3 in sd_event_add_time_relative (e=0xf7ffd3f0, ret=0xf7b01fa0, clock=1, usec=10000, accuracy=0, callback=0x565564ad <inner_timer_handler>, userdata=0xf7b01fa4) at ../src/libsystemd/sd-event/sd-event.c:1455
t = 17868423377094431728
r = <optimized out>
#1 0x565574c7 in sd_event_run_timer_fiber (userdata=0x0) at ../src/libsystemd/sd-event/test-event-future.c:329
inner = 0xf7ffd3f0
source = 0x0
counter = 0
r = <optimized out>
#2 0xf7dab308 in fiber_entry_point () at ../src/libsystemd/sd-future/fiber.c:208
_cleanup_log_unset_prefix_5 = 0x0
__unique_prefix_c6 = 0x5655bb30
f = 0xf7b06840
__func__ = "fiber_entry_point"
fake_stack_save = <optimized out>
#3 0xf7b032ae in setcontext () from /lib/libucontext.so.1
No symbol table info available.
#4 0x00000000 in ?? ()
No symbol table info available.
```
Building systemd with -mstackrealign makes all observed failures
disappear, suggesting a stack alignment issue in the interaction
between compiler-generated code and the ucontext implementation.
This resembles historical stack alignment issues in glibc's i386
makecontext() implementation, which required fixes to preserve the
expected stack alignment.
As a workaround, add -mstackrealign when building on i386 with musl
and libucontext.
Suggested-by: ChatGPT (OpenAI)
```
../src/boot/test-efi-string.c: In function 'test_xvasprintf_status':
../src/boot/test-efi-string.c:744:34: error: format '%zi' expects argument of type 'signed size_t', but argument 4 has type 'long int' [-Werror=format=]
744 | test_printf_one("%i %i %zi", INT_MIN, INT_MAX, SSIZE_MAX);
| ~~^
| |
| int
| %li
cc1: some warnings being treated as errors
ninja: subcommand failed
```
After enumeration, networkd may receive RTM_NEWLINK messages carrying a
stale interface name. This can happen when interface rename notifications
are queued before link enumeration and processed afterwards.
Previously, networkd could become confused by such a message and put the
corresponding Link into the failed state. Avoid this by checking whether
the new interface name is already in use by another interface and ignoring
the rename if so.
Fixes#20203.
E.g. on issue #20203, we may wrongly remove entry for another interface.
Let's mitigate issue on such situation for safety, though this does not
solve the issue.
This reverts commit 176b8be10f.
The check introduced by the commit is racy, as when format_ifname() is
called, the interface may be already renamed (again) to another name.
This is typically problematic when we swap interface names:
```
ip link set aaa name tmpname
ip link set bbb name aaa
ip link set tmpname bbb
```
When networkd received the notification about name change aaa -> tmpname,
the interface is already renamed from tmpname to bbb. So, the reverted
logic skips updating Link.ifname:
```
aaa: New interface name 'tmpname' received from the kernel does not correspond with the name currently configured on the actual interface 'bbb'. Ignoring.
```
On the second notification about name change bbb -> aaa, networkd fails to
update the mapping Manager.links_by_name, as we skipped previous renaming
and the mapping still has an outdated entry for 'aaa'.
```
bbb: Interface name change detected, renamed to aaa.
aaa: Failed to manage link by its new name: File exists
aaa: Could not process link message: File exists
aaa: Failed
aaa: State changed: configured -> failed
```
By reverting the commit, the issue is fixed. But the commit intended to
fix another issue #20203. So this reintroduces #20203. Let's fix it
in a later commit.
Fixes#42527.
Reintroduces #20203.
When a slot is disconnected, previously we tried to remove the slot from
the hashmap with a wrong key. Hence, the pointer to the freed slot object
remained in the hashmap.
When libdevmapper is built without UDEV_SYNC_SUPPORT (e.g. on Alpine/postmarketOS),
it creates a device node under /dev/mapper/ instead of relying on udev to create a symlink.
Yu Watanabe