This does not yet support directory images properly
as systemd itself does not support unpriv directory
images properly yet.
The user profiles are a copy of the system profiles but without
DynamicUser=yes (can't be used by user managers) and without
ProtectHome=yes (this masks /home which breaks StateDirectory= which
is lcoated inside /home)
Currently, our test logs are flooded with useless NSS varlink debug
logs coming from nss-systemd talking to each varlink userdb service
individually. Let's set SYSTEMD_NSS_LOG_LEVEL=info to get rid of these
verbose logs.
In order to make "freeze" attacks against the update logic harder let's
add the ability to encode a "Best Before" date into SHA256SUMS directory
listings: if the current time is already beyond that time, we'll ignore
the SHA256SUMS as "stale" and fail the upgrade. Or in other words: the
freeze attack will now result in a client-side error eventually, instead
of success state.
The best before data is encoded in an optional pseudo-file listed in SHA256SUMS:
any file named BEST-BEFORE-YYYY-MM-DD.
Add an option to generate the expected volume key hash for LUKS volumes
by systemd-repart
and put it to crypttab, make systemd-cryptsetup check it upon attaching.
The format of the hash
matches what's currently being measured to TPM2 PCR with
tpm2-measure-pcr=.
Closes#40123
The test checks that the expected hash is correctly recorded to the
generated crypttab and also checks that systemd-cryptsetup handles
the option correctly.
When attaching images generate a policy in the portable drop-in
that matches the partition types and content found while dissecting,
so that it can no longer be changed later without a reattach.
Main focus was to not introduce breaking change or duplicated argument.
The `--limit-messages=` option that is used under the `monitor` verb is
reused here. However, both `wait` and `monitor` have contradicting
default behaviors, so it's not the cleanest thing to do.
There was some post-commit discussion about the API in #33961, but the
final name adopted in #34928 wasn't that flexible either to fit nicely
here in the `wait` verbe.
Additionally, there wasn't consideration in #34555 for having uniform
behavrious, so we ended with `wait` verb and `--limit-messages=` option,
rather than `receive` verb with default of continuously receving signals
withtout exit so `--limit-messages=` make more sence and be expectable.
RootImageOptions=/ExtensionImages=/MountImages= all support custom
mount options, but mountfsd does not support it. Add varlink
parameters to allow callers to specify mount options so that
those directives can work as expected. Require additional privs via
polkit.
This is in preparation for later changes. The 1500+ line switch statement was very
hard to work with and had quite a bit of duplicated code. This is a rework that
splits outs parts into functions. I split this up into a bunch of commits so that it
can be reasonably reviewed. When viewed as one giant commit, diff just shows
a giant block of removed code and a giant block of added code.
In some cases I wasn't sure why one or the other of the identity arrays is used…
I preserved existing behaviour in all cases.
When creating ext2/ext3/ext4 filesystems, mke2fs generates a random
hash_seed for htree directory indexing. This causes non-reproducible
images even when SOURCE_DATE_EPOCH is set and the same filesystem UUID
is used.
Set the hash_seed explicitly to match the filesystem UUID, ensuring
that repeated builds with the same seed produce bit-for-bit identical
images.
Also add a test case in TEST-58-REPART to verify ext4 reproducibility
by creating the same partition twice and comparing the results.
See https://vdwaa.nl/mkosi-reproducible-arch-images.html
I used claude ai to help me with this change.
RootImageOptions=/ExtensionImages=/MountImages= all support
custom mount options, use the new mountfsd parameters to
configure them if they are specified.
This requires additioanl privileges via polkit due to security
implications of mount options, so document an example policy
that allows to use the nosuid mount option.
Currently mount options are handled in such a way that the first
definition for a given partition wins, and documented as such.
Change them so that they behave like other options, and the
last specified wins.
Applies to RootImageOptions=, MountImages= and ExtensionImages=.
Switch from a linked list to an array indexed by the partition
specifier to store them.
As announced by a few releases now, finally drop support for
sysvinit scripts.
Keep rc-local generator for now, as it's really a distinct
feature even though from the same era.
Otherwise, if the VM is unexpectedly rebooted, then `importctl --user pull-tar`
may fail as the file may already exist.
```
[ 123.351751] TEST-13-NSPAWN.sh[3946]: + run0 -u testuser importctl --user pull-tar file:///var/tmp/image-tar/kurps.tar.gz nurps --verify=checksum -m
[ 123.541603] TEST-13-NSPAWN.sh[4311]: Enqueued transfer job 3. Press C-c to continue download in background.
[ 123.552456] TEST-13-NSPAWN.sh[4311]: Pulling 'file:///var/tmp/image-tar/kurps.tar.gz', saving as 'nurps'.
[ 123.552788] TEST-13-NSPAWN.sh[4311]: Operating on image directory '/home/testuser/.local/state/machines'.
[ 123.819942] TEST-13-NSPAWN.sh[4311]: Got 1% of file:///var/tmp/image-tar/kurps.tar.gz.
[ 124.156557] TEST-13-NSPAWN.sh[4311]: * shutting down connection #0
[ 124.156896] TEST-13-NSPAWN.sh[4311]: * Could not open file /var/tmp/image-tar/kurps.tar.gz.sha256
[ 124.157223] TEST-13-NSPAWN.sh[4311]: * closing connection #-1
[ 124.159198] TEST-13-NSPAWN.sh[4311]: * Could not open file /var/tmp/image-tar/kurps.nspawn
[ 124.159493] TEST-13-NSPAWN.sh[4311]: * closing connection #-1
[ 124.159818] TEST-13-NSPAWN.sh[4311]: Acquired 68.5M.
[ 124.160395] TEST-13-NSPAWN.sh[4311]: Download of file:///var/tmp/image-tar/kurps.tar.gz complete.
[ 124.160664] TEST-13-NSPAWN.sh[4311]: Transfer failed: Could not read a file:// file
[ 124.160923] TEST-13-NSPAWN.sh[4311]: Settings file could not be retrieved, proceeding without.
[ 124.404733] TEST-13-NSPAWN.sh[4311]: * shutting down connection #1
[ 124.405162] TEST-13-NSPAWN.sh[4311]: Acquired 79B.
[ 124.406170] TEST-13-NSPAWN.sh[4311]: Download of file:///var/tmp/image-tar/SHA256SUMS complete.
[ 124.406734] TEST-13-NSPAWN.sh[4311]: SHA256 checksum of file:///var/tmp/image-tar/kurps.tar.gz is valid.
[ 124.455446] TEST-13-NSPAWN.sh[4311]: Failed to rename to final image name to /home/testuser/.local/state/machines/.tar-file:\x2f\x2f\x2fvar\x2ftmp\x2fimage-tar\x2fkurps\x2etar\x2egz: File exists
[ 124.457251] TEST-13-NSPAWN.sh[4311]: Exiting.
```
Workaround for issue #38240.
The verity user certs weren't looked up in the given --root= for
systemd-sysext which made it fail to set up extensions with a strict
image policy.
Look up verity user certs from inside the --root= when we operate on
images in it. The main use case where this matters is when the initrd
sets up the extensions for the final system and thus systemd-sysext
should do the same thing as it would do in the final system.
The test passes but for two minutes the cleanup function just logs:
Failed to inspect home: No home for user homedsshtest known
Failed to inspect home: No home for user homedsshtest known
Failed to inspect home: No home for user homedsshtest known
...
Do not wait for the user to become inactive or remove the user if 'homectl
inspect' doesn't find the user. This brings a successfull run from 176 s
to 58 s here.
With a service with DynamicUser= with static user or group, e.g.,
```
$ systemd-run -p DynamicUser=yes -p Group=disk sleep infinity
```
previously the lookup by name and ID through DBus/Varlink are inconsistent:
```
$ busctl call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager LookupDynamicUserByUID "u" 6
Call failed: Dynamic user ID 6 does not exist.
$ busctl call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager LookupDynamicUserByName "s" disk
u 6
$ userdbctl group 6
Group name: disk
Disposition: system
GID: 6
Passwords: 1
Service: io.systemd.NameServiceSwitch
$ userdbctl group disk
Group name: disk
Disposition: dynamic
GID: 6
Description: Dynamic Group
Service: io.systemd.DynamicUser
```
With this change, the results of these methods are consistent.
Fixes#40228.
MultiPathRoute= can now specify device-only nexthops without a gateway
address, e.g. MultiPathRoute=@wg0. This enables ECMP configurations over
interfaces that don't use gateway addresses, such as WireGuard tunnels.
The syntax is extended from "address[@device] [weight]" to
"[address]@device [weight]". The address is now optional, but at least
one of gateway or device must be specified. The @ symbol must still be
present for device-only routes, making the syntax unambiguous: @wg0
specifies a device, while a bare IP address specifies a gateway.
Device-only nexthops are only available for IPv4 routes. Device-only
multipath routes for IPv6 are not supported by the kernel's netlink
interface and will be rejected with a warning.
This change is fully backwards compatible. All existing configurations
continue to work unchanged, as they always included a gateway address.
Closes#39699.
Allow CPUSchedulingPolicy to be set to "ext". SCHED_EXT is a new
scheduling policy in Linux v6.12 that allows processes to be scheduled
using custom BPF schedulers instead of the default in-kernel ones.
Selectively setting the SCHED_EXT policy is useful for systems running
in "partial mode" where not all processes are run using a custom
scheduler.
Fallback to SCHED_OTHER and print an error message for systems where
SCHED_EXT isn't available.
Loading images is, generally speaking, the slowest part of sd-executor
when spawning a service. This is due to multiple factors. dm-verity is
obviously a big part of the cost, but dissecting in general via libblkid
also can take a lot of time, due to probing the images and their filesystems.
A performance test doing service restarts in a row shows these
results, ran on a production system (low power and slow ARM64 SOC) with
a real production service, show the following service interruption intervals:
Count | P50 (ms) | P90 (ms) | P95 (ms) | P99 (ms) | P99.9 (ms)
507 | 1,532 | 1,726 | 2,548 | 4,112 | 4,592
One iteration is 507 restarts in this case, but this has ran hundreds
of times and the results are always in line within margin of error.
This also holds true for metrics from live systems, same numbers.
Between 1.0s and 1.2s can be attributed by profiling to the time needed
for the service code itself to start up and sd_notify, the rest is spent
inside systemd's code.
This means there is currently a tradeoff for services - either use secure
images, or make restarting fast. Downtime of services is a very important
metric, as for many cases this directly translates to outages, total or
partial (blackouot or greyout).
In order to facilitate using secure images without downsides, skip the
slow dissect steps (probing, loop devices, etc) when the configured
image is a single filesystem dm-verity image with a policy that pins it
to a single filesystem, and an already existing and open dm-verity device
can be found and reused.
This allows orchestrators to pre-open images on download, before restarting
services, to minimize downtimes.
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
Authenticated disk encryption is experimentally supported by cryptsetup
since v2.0.0 and allows for automatic dm-integrity setup for LUKS
devices. Add support for the mode to systemd-repart. The PR adds support
for `cryptsetup luksFormat --integrity` to systemd-repart and
"encryptedwithintegrity" dissection policy.
Limitations:
- No discard, online-only mode for repart.
Closes#39250
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
Daan De Meyer
Lennart Poettering
Yu Watanabe
Nick Rosbrook
Vitaly Kuznetsov
Luca Boccassi
zefr0x
Mike Yuan
Heran YangandJian Wen
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
ZauberNerd
Usama Arif
Cathy Hu
Kai Lueke
Luiz Amaral
Matt Fleming