Currently only a single job for a single unit can be enqueued atomically,
so there is no guarantee that, e.g., starting a unit and its socket
at the same time will happen in the same transaction. That forces
callers to 'know' the right order in which to start new units being
installed, or failures will occur. It also means some ordering
constraints are ignored, in case the separate calls are done
in the wrong manual order.
Add a new EnqueueUnitJobMany() D-Bus method that takes a list of units
to start.
Fixes https://github.com/systemd/systemd/issues/8102
Co-authored-by: Michal Koutný <mkoutny@suse.com>
The test occasionally fails on GHA CI when formatting with xxhash64
because dm-integrity's crypto_alloc_shash() -> request_module() path
flakily fails to load the algorithm:
[ 29.172664] TEST-67-INTEGRITY.sh[447]: + for a in crc32c crc32 xxhash64 sha1 sha256
[ 29.172664] TEST-67-INTEGRITY.sh[447]: + [[ xxhash64 == crc32 ]]
[ 29.172664] TEST-67-INTEGRITY.sh[447]: + test_one xxhash64 0
[ 29.172664] TEST-67-INTEGRITY.sh[447]: + integritysetup format /dev/loop0 --batch-mode -I xxhash64 ''
[ 29.223383] TEST-67-INTEGRITY.sh[1220]: device-mapper: reload ioctl on temporary-cryptsetup-fa8bebe3-1d87-4796-91e8-abc02c487bb5 (254:0) failed: No such file or directory
[ 29.226916] kernel: device-mapper: table: 254:0: integrity: Invalid internal hash (-ENOENT)
[ 29.227415] kernel: device-mapper: ioctl: error adding target to table
[ 29.231586] TEST-67-INTEGRITY.sh[1220]: Cannot format integrity for device /dev/loop0.
Preload each algorithm's crypto module before use, and skip algorithms
that are not registered in /proc/crypto.
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
```
TEST-02-UNITTESTS.sh[4382]: [ 707.393188] test-cgroup-util[426]: Failed to open pidfd for pid 414: Invalid argument
TEST-02-UNITTESTS.sh[4382]: [ 707.393193] test-cgroup-util[426]: src/test/test-cgroup-util.c:249: Assertion failed: Expected "r = proc_dir_read_pidref(d, &pid)" to succeed, but got error: -22/EINVAL
```
The kernel can return EINVAL on pidfd_open() when the selected PID is
not a thread group leader. Don't fail the test, as we are iterating on
everything, so this can seldomly happen.
When attaching images generate a policy in the portable drop-in that
matches the partition types and content found while dissecting, so that
it can no longer be changed later without a reattach
RootImageOptions=/ExtensionImages=/MountImages= all support custom
mount options, but mountfsd does not support it. Add varlink
parameters to allow callers to specify mount options so that
those directives can work as expected. Require additional privs via
polkit.
Loading images is, generally speaking, the slowest part of sd-executor
when spawning a service. This is due to multiple factors. dm-verity is
obviously a big part of the cost, but dissecting in general via libblkid
also can take a lot of time, due to probing the images and their
filesystems.
A performance test doing service restarts in a row shows these
results, ran on a production system (low power and slow ARM64 SOC) with
a real production service, show the following service interruption
intervals:
```
Count | P50 (ms) | P90 (ms) | P95 (ms) | P99 (ms) | P99.9 (ms)
507 | 1,532 | 1,726 | 2,548 | 4,112 | 4,592
```
One iteration is 507 restarts in this case, but this has ran hundreds
of times and the results are always in line within margin of error.
This also holds true for metrics from live systems, same numbers.
Between 1.0s and 1.2s can be attributed by profiling to the time needed
for the service code itself to start up and sd_notify, the rest is spent
inside systemd's code.
This means there is currently a tradeoff for services - either use
secure
images, or make restarting fast. Downtime of services is a very
important
metric, as for many cases this directly translates to outages, total or
partial (blackouot or greyout).
In order to facilitate using secure images without downsides, skip the
slow dissect steps (probing, loop devices, etc) when the configured
image is a single filesystem dm-verity image with a policy that pins it
to a single filesystem, and an already existing and open dm-verity
device
can be found and reused.
This allows orchestrators to pre-open images on download, before
restarting
services, to minimize downtimes.
Usually we gracefully ignore unknown configuration parameters, so that
service files can be written by upstreams and used across a variegated
range of distributions with various versions of systemd, to avoid
forcing users to the minimum common denominator and only adding settings
that are supported by the oldest distro supported.
Image policies do not behave like this, and any unknown partition or
policy designator causes the whole unit to fail to parse and a hard
error.
Change it so that parsing RootImagePolicy and friends via unit file or
D-Bus logs but otherwise ignores unknown specifiers, like other options
do.
This allows us to add new specifiers in the future, and users to adopt
them immediately.
Follow-up for d452335aa4
dnsmasq 2.92 started converting binary content to ascii so the previous
invalid URL is no longer invalid as it's escaped.
So the test started failing: https://bugs.debian.org/1122015
Use a '|' character instead, which is not a valid URL character but it's
also not escaped by dnsmasq.
Follow-up for 1219391c9f
This updates example output in systemd-analyze's man page after the
tool's output was changed in a previous commit.
Additionally bash completion is added for `systemd-analyze filesystems`
and improved for `systemd-analyze calendar`.
Currently the only supported integrity algorithm using HMAC is
`hmac-sha256`. Add `hmac-sha512` to the list of supported algorithms as
well.
Also add the `PHMAC` integrity algorithm to the list of supported
algorithms. The `PHMAC` algorithm is like the regular HMAC algorithm,
but it takes a wrapped key as input. A key for the `PHMAC` algorithm is
an opaque key blob, who's physical size has nothing to do with the
cryptographic size. Such a wrapped key can for example be a HSM
protected key. Currently PHMAC is only available for the s390x
architecture (Linux on IBM Z).
Support for PHMAC has just been added to the cryptsetup project via MR
https://gitlab.com/cryptsetup/cryptsetup/-/merge_requests/693 by commit
https://gitlab.com/cryptsetup/cryptsetup/-/commit/296eb39c60bd85228930caaea4bb1d2a766b7544
To allow automatic opening of integrity protected volumes that use PHMAC
via `/etc/integritytab`, this change in systemd's integritysetup tool is
needed as well.
Before calling io.systemd.MachineImage.List.
The systemd-nspawn process takes a lock in the run() function in
nspawn.c and holds it for the entire runtime of that function. If we
call `machinectl terminate` the machine gets unregistered _before_ we
release the lock, so the original `machinectl status` check would return
early, allowing for a race where we call io.systemd.MachineImage.List
over Varlink when systemd-nspawn still holds the lock because the
process is still running.:
```
[ 41.691826] TEST-13-NSPAWN.sh[1102]: + machinectl terminate long-running
[ 41.695009] systemd-nspawn[2171]: Trying to halt container by sending TERM to container PID 1. Send SIGTERM again to trigger immediate termination.
[ 41.698235] systemd-machined[1192]: Machine long-running terminated.
[ 41.709520] TEST-13-NSPAWN.sh[1102]: + systemctl kill --signal=KILL systemd-nspawn@long-running.service
[ 41.709169] systemd-nspawn[2171]: Failed to unregister machine: No machine 'long-running' known
[ 41.720869] TEST-13-NSPAWN.sh[2346]: + varlinkctl --more call /run/systemd/machine/io.systemd.MachineImage io.systemd.MachineImage.List '{}'
[ 41.723359] TEST-13-NSPAWN.sh[2347]: + grep long-running
...
[ 41.735453] TEST-13-NSPAWN.sh[2352]: + varlinkctl call /run/systemd/machine/io.systemd.MachineImage io.systemd.MachineImage.List '{"name":"long-running", "acquireMetadata": "yes"}'
[ 41.736222] TEST-13-NSPAWN.sh[2353]: + grep OSRelease
[ 41.739500] TEST-13-NSPAWN.sh[2352]: Method call io.systemd.MachineImage.List() failed: Device or resource busy
[ 41.740641] systemd[1]: Received SIGCHLD.
[ 41.740670] systemd[1]: Child 2171 (systemd-nspawn) died (code=killed, status=9/KILL)
[ 41.740725] systemd[1]: systemd-nspawn@long-running.service: Child 2171 belongs to systemd-nspawn@long-running.service.
[ 41.740748] systemd[1]: systemd-nspawn@long-running.service: Main process exited, code=killed, status=9/KILL
[ 41.740755] systemd[1]: systemd-nspawn@long-running.service: Will spawn child (service_enter_stop_post): systemd-nspawn
[ 41.740872] systemd[1]: systemd-nspawn@long-running.service: About to execute: systemd-nspawn --cleanup --machine=long-running
...
```
Let's mitigate this by waiting until the corresponding
systemd-nspawn@.service instance enters the 'inactive' state where the
lock should be properly released.
Resolves: https://github.com/systemd/systemd/issues/39547
Needed to implement support for RootHashSignature=/RootVerity=/RootHash=
and friends when going through mountfsd, for example with user units,
so that system and user units provide the same features at the same
level
https://github.com/systemd/systemd/pull/39168 made verity sharing
opt-in, and enabled it for system services.
Also enable it for user services for RootImage/etc, and for nspawn, for
the same reasons.
Registering a process as a machine means a caller can get machined to
send sigterm to it, and more. If an unpriv user is registering, ensure
the registered process has the same uid.
Follow-up for adaff8eb35
There is a botched arm64 linker transition going on, where a new feature
is enabled (GCS) and the linker fails the build unless all object files
being linked are built with the new specific feature. This was enabled
in the toolchain (GCC 15) _before_ all libraries were rebuilt, including
glibc, so everything fails. The toolchain maintainers declined to fix it
and instead say that this is a useless warning to have, and to just
disable it and ignore it.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1110461
> systemd fails to build from source on arm64 if built with GCC-15, currently in
> experimental.
>
> GCC-15 includes support for an arm64 security feature called Guarded Control
> Stack (GCS). To help with GCS adoption, the linker warns about shared libraries
> built without GCS. For example:
>
> /usr/lib/gcc/aarch64-linux-gnu/15/../../../aarch64-linux-gnu/libncursesw.so.6:
> warning: GCS is required by -z gcs, but this shared library lacks the necessary property note.
> The dynamic loader might not enable GCS or refuse to load the program unless
> all the shared library dependencies have the GCS marking.
>
> The warning is harmless, and can be ignored. However, systemd is built with
> --fatal-warnings, and for this reason will fail to build on arm64 once GCC-15
> becomes the default compiler in Debian.
[585/3230] Linking target src/core/libsystemd-core-258.so
FAILED: src/core/libsystemd-core-258.so
gcc -o src/core/libsystemd-core-258.so -Wl,--as-needed -Wl,--no-undefined -shared -fPIC -Wl,-soname,libsystemd-core-258.so -Wl,--whole-archive -Wl,--start-group src/core/libsystemd-core-258.a -Wl,--no-whole-archive -fstack-protector -Wl,-z,relro -specs=/usr/share/debhelper/dh_package_notes/debian-package-notes.specs -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/work/src=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -mbranch-protection=standard -O0 -g -Og -Wdate-time '-Wl,-rpath,$ORIGIN/../shared' src/shared/libsystemd-shared-258.so -shared -Wl,--version-script=/work/src/src/shared/libshared.sym /usr/lib/aarch64-linux-gnu/libacl.so /usr/lib/aarch64-linux-gnu/libaudit.so /usr/lib/aarch64-linux-gnu/libblkid.so -ldl -lm /usr/lib/aarch64-linux-gnu/libmount.so -lrt /usr/lib/aarch64-linux-gnu/libseccomp.so /usr/lib/aarch64-linux-gnu/libselinux.so -Wl,--end-group -pthread -Wl,--fatal-warnings -Wl,-z,now -Wl,-z,relro -Wl,--gc-sections -Wl,--warn-common
src/shared/libsystemd-shared-258.so: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/usr/lib/aarch64-linux-gnu/libacl.so: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/usr/lib/aarch64-linux-gnu/libaudit.so: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/usr/lib/aarch64-linux-gnu/libblkid.so: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/lib/aarch64-linux-gnu/libm.so.6: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/lib/aarch64-linux-gnu/libmvec.so.1: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/usr/lib/aarch64-linux-gnu/libmount.so: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/usr/lib/aarch64-linux-gnu/libseccomp.so: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/usr/lib/aarch64-linux-gnu/libselinux.so: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/lib/aarch64-linux-gnu/libc.so.6: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
/lib/ld-linux-aarch64.so.1: warning: GCS is required by -z gcs, but this shared library lacks the necessary property note. The dynamic loader might not enable GCS or refuse to load the program unless all the shared library dependencies have the GCS marking.
collect2: error: ld returned 1 exit status
mkosi patches up /etc/os-release to add local IDs and fixup certain
issues, so when tests patch /usr/lib/ on the fly, copy to the version in
/etc/ too to avoid test failures when querying
6370s 10/98 systemd:integration-tests / TEST-07-PID1 FAIL 31.03s exit
status 1
6370s 25/98 systemd:integration-tests / TEST-29-PORTABLE FAIL 12.76s
exit status 1
6370s 33/98 systemd:integration-tests / TEST-43-PRIVATEUSER-UNPRIV FAIL
6.57s exit status 1
6370s 37/98 systemd:integration-tests / TEST-50-DISSECT FAIL 16.97s exit
status 1
This is particularly an issue when running these tests on debian unstable,
where mkosi has to fixup os-release to make it valid and avoid further
breakages:
https://github.com/systemd/mkosi/blob/main/mkosi/distributions/debian.py#L234
This extends the functionality introduced in #21648 to allow using
addresses chosen from a delegated prefix as well as the existing
SLAAC/LL/DHCP functionality.
Enable nspawn job, as there's no nested kvm so VMs are too slow. Fix
some tests that fail in a VM anyway, might add a nightly job later that
runs them.
This makes it possible to build systemd with
-Dc_args='-DOPENSSL_NO_UI_CONSOLE=1'. Hopefully, now systemd can be
built with other openssl implementations, like BoringSSL, which deos not
support UIs.
Closes#38024.
* b2fe7ba6b1 Do not use return in finally
* b4f3088535 Ensure directories exist before running sync scripts
* e83e7036b0 mkosi-obs: also allow enrolling additional certs in KEK
* b6432fbc29 initrd: Copy /etc/pki/rpm-gpg into sandbox tree
* 44b96b93c8 Partially revert "resources: add systemd-boot-efi"
* 7a22260179 Check that keys are both in context.cli and history
* 6570b21a4b fix: Remove `erofs-utils` from initrd
* ff61c5022a mkosi-initrd: Add atkbd and i8042 modules to the default
initrd modules
* eaa37f7c6c Generate initrds outside of umask context manager
* de09209520 ci: Drop differential shellcheck
* d07c6b93c9 build(deps): bump github/codeql-action from 3.28.16 to
3.28.18
* 9a12fb1bef Check that commit exists in config_parse_minimum_version()
* 9e5b4f8932 kmod: factor out comparison and also count empty
directories as invalid
* 39ed5ef6c2 apt: support including additional configuration for
reprepro
The pkcs7_new() function currently uses the hash algorithm from the X509
certificate's signature, but the content signature may use a different
hash algorithm. This adds a parameter to allow specifying what hash
algorithm the content signature generation used.
NetworkManager imports some systemd code, which we then build with
`-Wcast-align=strict`. This results in this warning:
```
../src/libnm-systemd-shared/src/basic/pidfd-util.c: In function 'pidfd_get_inode_id':
../src/libnm-systemd-shared/src/basic/pidfd-util.c:255:41: warning: cast increases required alignment of target type [-Wcast-align]
255 | *ret = *(uint64_t*) fh.file_handle.f_handle;
| ^
```
Since the address of `f_handle` should always be aligned at the word
boundary (it is preceded by 2 int`s` in `file_handle`), this patch uses
`CAST_ALIGN_PTR()` to avoid this warning.
If --generate-fstab=PATH is used, there is the possibility that the
fstab file already exists, making systemd-repart fail.
This commit will add a new --append-fstab= parameter, that will read
the file and merge it with the new generated content. Using the
comments, the command can separate the automatic-generated section from
the user-provided section, allowing for the next append the replacement
only of the automatic-generated section, keeping the user one.
`ExtensionImages=` and `ExtensionDirectories=` now let you specify
vpick-named extensions; however, since they just get set up once when
the service is started, you can't see newer versions without restarting
the service entirely. Here, also reload confext extensions when you
reload a service. This allows you to deploy a new version of some
configuration and have it picked up at reload time without interruption
to your workload.
Right now, we would only reload confext extensions and leave the sysext
ones behind, since it didn't seem prudent to swap out what is likely
program code at reload. This is made possible by only going for the
`SYSTEMD_CONFEXT_HIERARCHIES` overlays (which only contains `/etc`).
This PR:
- Adjusts `service.c` to also refresh extensions when needed.
- Adds integration tests to check that a confext reload actually
occurred.
- Adds to the `systemd.exec` man pages to document this behavior.
This is a follow up to #24864 and #31364. Thank you to @bluca and
@goenkam for help in getting this up.
Update hwdb, even though we are pretty far from a release in order to:
- incrementally test upstream additions to hwdb
- allow backports to stable branches
Luca BoccassiandMichal Koutný
