Loading images is, generally speaking, the slowest part of sd-executor
when spawning a service. This is due to multiple factors. dm-verity is
obviously a big part of the cost, but dissecting in general via libblkid
also can take a lot of time, due to probing the images and their
filesystems.
A performance test doing service restarts in a row shows these
results, ran on a production system (low power and slow ARM64 SOC) with
a real production service, show the following service interruption
intervals:
```
Count | P50 (ms) | P90 (ms) | P95 (ms) | P99 (ms) | P99.9 (ms)
507 | 1,532 | 1,726 | 2,548 | 4,112 | 4,592
```
One iteration is 507 restarts in this case, but this has ran hundreds
of times and the results are always in line within margin of error.
This also holds true for metrics from live systems, same numbers.
Between 1.0s and 1.2s can be attributed by profiling to the time needed
for the service code itself to start up and sd_notify, the rest is spent
inside systemd's code.
This means there is currently a tradeoff for services - either use
secure
images, or make restarting fast. Downtime of services is a very
important
metric, as for many cases this directly translates to outages, total or
partial (blackouot or greyout).
In order to facilitate using secure images without downsides, skip the
slow dissect steps (probing, loop devices, etc) when the configured
image is a single filesystem dm-verity image with a policy that pins it
to a single filesystem, and an already existing and open dm-verity
device
can be found and reused.
This allows orchestrators to pre-open images on download, before
restarting
services, to minimize downtimes.
So far we compiled the normal and standalone versions completely
independently. Let's use the 'extract' template pattern to avoid any
additional compilation and only require an single link to produce the
.standalone variants.
Unfortunately, as designed, the 'extract' framework only allows one set of
object files to be extracted. Since we need all the files for the
.standalone version, we cannot use 'extract' for other purposes. Thus, in
the two cases where 'extract' was used for the test binaries, this is now
changed to compile the files a second time. But the number of files in that
list is small, so this seems like a better option.
(If we weren't using the template system, we could easily extract just the
objects we need. But with the current system, at the point of the
definition, the binaries are not defined yet. We'd need to handle all of
this through sets of dictionaries, and that just seems like too much
trouble to avoid double compilation of a few small files.)
If we ever want to add it back, it should be with -DSTANDALONE=0|1, so
that #if instead of #ifdef can be used. We generally converted our internal
defines to that form.
This effectively reverts 3537577c37. Originally,
the #ifdefs were added because we didn't want to pull in the whole tree of
libmount and other dependencies in standalone versions. But dependencies are
now loaded through dlopen(), so this is not needed anymore. (And doesn't even
make much of a difference.)
Allow .standalone version to be built on-demand, even if -Dstandalone=false
is configured. In other words, this changes the meson option from a hard
disablement to a soft "build is on/off by default".
The meson config was originally written in this way but we lost this feature
after the transition to templates. It is nice to build additional targets on
demand during development, so add this back.
The indirection through variables doesn't seem that useful here:
OTOH, the lists are short, and OTOH, there is a bunch of different
programs with similar names. Overall, it's all easier to follow if
the lists are inline.
* cac8dde28a test: Allow passing in extra tests to skip via TEST_SKIP
* 56377438ba Disable sysinit-path for upstream builds
* 0c8ea706f9 Fix links to patches
* 4f5b5a9615 Version 259
* bf8019c840 Version 259~rc3
* ef777d6572 Check if --max-lines is supported by meson
* b562e38e22 Fix use of removed $LOCAL_CONF variable
* 0289127dae Patch machined to continue after selinux denial
* 7e409130ee Version 259~rc2
* 33b38cdbc7 Suppress errors from tar
* ddb6474e94 Drop provides for removed sysvinit tools
* 9ac8c36307 Set meson auto features to auto when building for upstream
$ ./tools/fetch-distro.py -u fedora
+ mkosi --json -d fedora summary
‣ Ignoring --distribution from the CLI. Run with -f to rebuild the image with this setting
Follow-up for 35f6e5c327
Loading images is, generally speaking, the slowest part of sd-executor
when spawning a service. This is due to multiple factors. dm-verity is
obviously a big part of the cost, but dissecting in general via libblkid
also can take a lot of time, due to probing the images and their filesystems.
A performance test doing service restarts in a row shows these
results, ran on a production system (low power and slow ARM64 SOC) with
a real production service, show the following service interruption intervals:
Count | P50 (ms) | P90 (ms) | P95 (ms) | P99 (ms) | P99.9 (ms)
507 | 1,532 | 1,726 | 2,548 | 4,112 | 4,592
One iteration is 507 restarts in this case, but this has ran hundreds
of times and the results are always in line within margin of error.
This also holds true for metrics from live systems, same numbers.
Between 1.0s and 1.2s can be attributed by profiling to the time needed
for the service code itself to start up and sd_notify, the rest is spent
inside systemd's code.
This means there is currently a tradeoff for services - either use secure
images, or make restarting fast. Downtime of services is a very important
metric, as for many cases this directly translates to outages, total or
partial (blackouot or greyout).
In order to facilitate using secure images without downsides, skip the
slow dissect steps (probing, loop devices, etc) when the configured
image is a single filesystem dm-verity image with a policy that pins it
to a single filesystem, and an already existing and open dm-verity device
can be found and reused.
This allows orchestrators to pre-open images on download, before restarting
services, to minimize downtimes.
We should consider the whole init.scope trusted, and any
process trying to log on behalf of a unit there should be
attributed as so.
Follow-up for 4f25248b6e
Only with this commit is the change in effect.
* d9f2aa1704 Install systemd-tpm2-generator.8 only for UEFI builds
* ac1c7d8048 Drop dependencies on libcap-dev, no longer used since v259
* c36e5871ca Do not install systemd-sysv-generator.8 in upstream build
* bac0cca0e8 Install new files for upstream build
* 2855fb1302 Update changelog for 259-1 release
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
Authenticated disk encryption is experimentally supported by cryptsetup
since v2.0.0 and allows for automatic dm-integrity setup for LUKS
devices. Add support for the mode to systemd-repart. The PR adds support
for `cryptsetup luksFormat --integrity` to systemd-repart and
"encryptedwithintegrity" dissection policy.
Limitations:
- No discard, online-only mode for repart.
Closes#39250
The unix-domain socket to AF_VSOCK multiplexers in Firecracker and
vhost-device-vsock sends OK PORT response to the client, resulting
ssh clients to abort the connection with the additional response. This
patch addresses this issue by waiting and expecting the possible OK PORT
response from the multiplexer, if any, and then handover the socket fd
to the ssh client. It only checks if the response begins with OK and
consume the response till the first \n, for simplicity.
Signed-off-by: Miao Wang <shankerwangmiao@gmail.com>
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
When a Yubikey or other fido2 device has FIPS mode enabled, UV will
always be required and cannot be disabled. Unhelpfully, when it is not
sent down, the hardware token (not the library) returns a generic
FIDO_ERR_MISSING_PARAMETER:
$ systemd-cryptenroll --fido2-device=auto --fido2-with-client-pin=no foo
<...>
libfido2: rx: payload_len=1
libfido2: fido_rx: buf=0x55c9ce170940, len=1
libfido2: 0000: 14
libfido2: cbor_parse_reply: blob[0]=0x14
libfido2: fido_dev_get_assert_rx: adjust_assert_count
Failed to ask token for assertion: FIDO_ERR_MISSING_PARAMETER
This can be set even when the token doesn't support UV (as in, fingerprint
reader), in which case using the PIN is required.
Check if 'alwaysUv' is set, and if not already configured, force
either UV (if available) or PIN when enrolling.
Now that we have Management Component Transport Protocol (MCTP) transports
available over USB, it would be helpful to apply udev's persistent
naming rules to MCTP interfaces, to follow the USB hub/port topology.
Enable persistent naming for ARPHRD_MCTP-type devices, using a "mc" name
prefix, and add appropriate definitions for the v260 naming sheme.
Adds a new `SYSTEMD_NSS_RESOLVE_INTERFACE` environment variable to the nss-resolve module, whose value is subsequently passed down to the `ifindex` resolved lookup option.
This allows name lookups to be constrained to a just single interface for e.g. captive portal browsers.
This allows us to query the window size without moving the cursor. We have
various reports about the cursor being in an unexpected position and/or state.
Closes https://github.com/systemd/systemd/issues/37526.
Lennart Poettering
Daan De Meyer
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
Luca Boccassi
seidlerv
Mike Yuan
Miao Wang
Franck Bui
Jeremy Kerr
Popax21
Vitaly Kuznetsov