Loading images is, generally speaking, the slowest part of sd-executor
when spawning a service. This is due to multiple factors. dm-verity is
obviously a big part of the cost, but dissecting in general via libblkid
also can take a lot of time, due to probing the images and their filesystems.
A performance test doing service restarts in a row shows these
results, ran on a production system (low power and slow ARM64 SOC) with
a real production service, show the following service interruption intervals:
Count | P50 (ms) | P90 (ms) | P95 (ms) | P99 (ms) | P99.9 (ms)
507 | 1,532 | 1,726 | 2,548 | 4,112 | 4,592
One iteration is 507 restarts in this case, but this has ran hundreds
of times and the results are always in line within margin of error.
This also holds true for metrics from live systems, same numbers.
Between 1.0s and 1.2s can be attributed by profiling to the time needed
for the service code itself to start up and sd_notify, the rest is spent
inside systemd's code.
This means there is currently a tradeoff for services - either use secure
images, or make restarting fast. Downtime of services is a very important
metric, as for many cases this directly translates to outages, total or
partial (blackouot or greyout).
In order to facilitate using secure images without downsides, skip the
slow dissect steps (probing, loop devices, etc) when the configured
image is a single filesystem dm-verity image with a policy that pins it
to a single filesystem, and an already existing and open dm-verity device
can be found and reused.
This allows orchestrators to pre-open images on download, before restarting
services, to minimize downtimes.
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
Authenticated disk encryption is experimentally supported by cryptsetup
since v2.0.0 and allows for automatic dm-integrity setup for LUKS
devices. Add support for the mode to systemd-repart. The PR adds support
for `cryptsetup luksFormat --integrity` to systemd-repart and
"encryptedwithintegrity" dissection policy.
Limitations:
- No discard, online-only mode for repart.
Closes#39250
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
LUKS2 supports built-in integrity checking which may come very handy to
mitigate partial rollback attacks on the storage when only some specific
parts are restored to some old encrypted state. Specific use-cases like
Confidential VMs may want to mandate the usage of feature e.g. on the root
volume. Introduce "encryptedwithintegrity" image policy to support that.
Note, due to the current libcryptsetup limitations, checking whether the
feature is enabled or not for the 'file' case (e.g. DDI image as a raw file)
requires setting up a loop device. To avoid that and keep dissect fully
functional when working unpriviliged, implement a minimal custom LUKS header
parser.
There is currently no way to figure out a rate limit was hit on a unit,
as the last result is stripped in order to keep reporting the first
result, which is useful in case of a watchdog failure, which is the
reason why it was changed as such.
But rate limiting is also an important information to provide to
users, so allow the Result property to reflect it when it
happens.
PCR 16 is intended for debugging purposes, and is the only PCR that can
be reset by software. Let's prefer that for testing.
PCR 11 has a purpose (i.e. kernel is measured there on UKI systems, as
are the boot phases), hence let's not use that for testing.
That should remove any interference of boot time ordering and testing
our tpm code, as we will write to PCR 16 only, and reset it before we
rely on it.
This likely, kind of fixes#39582, but the issue mixes two things, PCR
11 issues on real-life hardware, and PCR 11 isues in our artificial
tests. Only the latter is addressed by this PR, the other issue needs
more debugging, but without logs we cannot do much.
Config files for --root= weren't picked up as expected because the
--root= flag got parsed after the config file.
Switch the order of config file and CLI flag parsing while letting the
CLI flags overwrite things set by the config files by tracking state
during parsing.
Usually we gracefully ignore unknown configuration parameters, so that
service files can be written by upstreams and used across a variegated
range of distributions with various versions of systemd, to avoid
forcing users to the minimum common denominator and only adding settings
that are supported by the oldest distro supported.
Image policies do not behave like this, and any unknown partition or
policy designator causes the whole unit to fail to parse and a hard
error.
Change it so that parsing RootImagePolicy and friends via unit file or
D-Bus logs but otherwise ignores unknown specifiers, like other options
do.
This allows us to add new specifiers in the future, and users to adopt
them immediately.
Follow-up for d452335aa4
When a TTY is attached to the test unit, grep -q will generate SIGPIPE
for the previous command in the pipeline which in combo with `pipefail`
will cause the command to fail with exit status 141 which will fail the
test.
Replace with >/dev/null to avoid this from happening.
See also https://www.gnu.org/software/grep/manual/html_node/Usage.html
> There is a related problem with Bash’s set -e -o pipefail. Since grep
> does not always read all its input, a command outputting to a pipe read
> by grep can fail when grep exits before reading all its input, and the
> command’s failure can cause Bash to exit.
Co-authored-by: Yu Watanabe <watanabe.yu+github@gmail.com>
Alpine/postmarketOS build coreutils with --enable-single-binary=symlinks.
In that case, all commands provided by coreutils are symlink to
/usr/bin/coreutils, and it calls prctl(PR_SET_NAME, argv[0]), hence the comm
will be the path to the symlink.
This also makes not kill sleep command with SIGKILL in TEST-17-UDEV, that is
totally unnecessary.
The test cases are not related to systemd-resolved.
While moving the test cases, now userdbctl is used for obtaining UID/GID
for the dynamic user, as musl does not support nss module, hence getent
does not provide information about the dynamic user.
If there is no corresponding audit session, then the session ID is
prefixed with 'c', and in that case the session ID should be used as is
in bus path.
If sh is not bash, some builtin command behave slightly differently.
E.g. if sh is provided by busybox, its builtin test command does not check if
the path is a mount point or not, and 'test -w' only checks the access mode of
the inode. So, even if a readonly filesystem is mounted on a directory,
the test command may succeed.
To avoid such confusion, let's unconditionally use bash.
in pcrextend we destroy pcr 11, and if we are booted in a kernel that
has pcr11 sigs, we cannot use that signature anymore. hence, let's do
the nvpcr test first, before doing the pcrextend stuff.
Fixes: #39582
After bdb8e584f4 we stopped rebasing the
next elapse timestamp unconditionally and the only case where we'd do
that was when both last trigger and last inactive timestamps were empty.
This covered timer units during boot just fine, since they would have
neither of those timestamps set. However, persistent timers
(Persistent=yes) store their last trigger timestamp on a persistent
storage and load it back after reboot, so the rebasing was skipped in
this case.
To mitigate this, check the last_trigger timestamp is older than the
current machine boot - if so, that means that it came from a stamp file
of a persistent timer unit and we need to rebase it to make
RandomizedDelaySec= work properly.
Follow-up for bdb8e584f4.
Resolves: #39739
We were getting a list of invocation IDs, picking one at random,
and then querying the unit. This is obviously racy.
TEST-74-AUX-UTILS.sh[2873]: + varlinkctl call /run/systemd/io.systemd.Manager io.systemd.Unit.List
'{"invocationID": "2052c9a5-7983-4f72-9910-c49e38c91dab"}'
TEST-74-AUX-UTILS.sh[3707]: Method call io.systemd.Unit.List() failed: io.systemd.Unit.NoSuchUnit
The complicated varlink + jq callout is replaced by a simple systemctl call.
I think that's better to avoid a complicated jq expression.
Fixes https://github.com/systemd/systemd/issues/38647.
We cannot just use %m, because strerror returns a confusing error message
for ESRCH or ENOEXEC. udev code was doing a good job, but the error handling
was very verbose. Let's encapsulate the customized error messages in a
helper.
No functional change, except that the error messages have a slightly different
form now. The old messages were a bit better, but we don't have as much
flexibility in the new scheme. "Failed to resolve user 'foo': Unknown user"
should be good enough.
Commit 88fce09026 modified the
mount_bind() function, causing it to perform arithmetic on the uid_shift
parameter. However, it performs this arithmetic even when uid_shift was
UID_INVALID, which was not intended. This typically occurred when
mount_custom() was called for a simple bind mount without user
namespaces (and thus no rootidmap mount option).
This arithmetic (e.g., uid_shift + m->destination_uid) then wraps
around, resulting in the invalid ID 4294967295 ((uid_t)-1).
This bug manifests for users running systemd-nspawn with
--link-journal=host and --volatile=yes (but without --private-users),
causing systemd-tmpfiles to fail.
Make mount_bind() robust by checking if uid_shift is valid before using
it in arithmetic. If it is UID_INVALID, it defaults to a shift of 0 for
the ownership calculation, restoring correct behavior for plain bind
mounts while preserving the intended logic for ID-mapped mounts.
Fixes: #39714
Luca Boccassi
Lennart Poettering
Daan De Meyer
Vitaly Kuznetsov
Mike Yuan
Yu Watanabe
Kai Lueke
Luca Boccassi
Chris Down
Frantisek Sumsal
Zbigniew Jędrzejewski-Szmek