This partially reverts 36c557f7d4, which
introduced notify_remove_fd() that logs in LOG_DEBUG. However,
notify_remove_fd_warn() is still called other library functions, e.g.
notify_push_fd(), and produces warning message about the failure in
removing fd from fdstore on shutdown.
During shutdown process, we get the following logs:
```
systemd-udevd[370]: Failed to send notify message to '/run/systemd/notify': Connection refused
systemd-udevd[370]: Failed to remove file descriptor "config-serialization" from the store, ignoring: Connection refused
systemd-udevd[370]: Failed to send notify message to '/run/systemd/notify': Connection refused
systemd-udevd[370]: Failed to push serialization fd to service manager: Connection refused
```
Here, the 1st, 3rd, and 4th messages are in LOG_DEBUG, but the 2nd one
was in LOG_WARNING before this commit, and this makes it also in LOG_DEBUG.
Follow-up for 472404aca5.
name_to_handle_at_try_unique_mntid_fid() in its current form is
ill-designed for various reasons:
* AT_HANDLE_FID requires file system support, while unique mount id
is a VFS concept hence is always available if supported. Hence
the fallback for AT_HANDLE_MNT_ID_UNIQUE should be independent
of fid.
* The request for AT_HANDLE_MNT_ID_UNIQUE can be identified via
specifying ret_unique_mnt_id, no need for opening up the control
to caller (and currently the function simply doesn't handle
mismatch between ret params and flags).
* The caller cannot realistically differentiate whether the returned
mount id is actually unique.
* The path_get_unique_mnt_id() fallback did not handle AT_SYMLINK_FOLLOW.
Let's instead move the statx() fallback into name_to_handle_at_loop()
directly, and revamp interaction of ret_mnt_id/ret_unique_mnt_id:
if both are set, it indicates that the caller can handle both, hence
set what we have and return 0/1 for whether we managed to acquire
the unique one.
The !ret_handle && ret_mnt_id logic is removed. Let's not rely on
undocumented bizaare behavior and it's unused anyways.
path_get_mnt_id_at() exists for a reason...
See the [design
doc](https://github.com/systemd/systemd/issues/38023#issue-3192567450)
by @ikruglov
This PR introduces the metrics API framework, adds some basic system
wide/per unit/service metrics, and a basic CLI. The PR is broken into
two commits as described below.
### Deviations from the original design
- Introduced top level field "object" for ease of filtering. Instead of
having `fields: { unit: "foo", unit_type: "service" }`, we now have
`object: foo.service` as the top level field.
### First commit
The first commit includes:
- Metrics API definitions
- Code to set up the varlink server
- The describe method which shows all the metrics families
- The list method which lists all the metrics
- Type definitions related to MetricFamily
- Common code to build json objects
### Second commit
The second commit adds some basic metrics, a basic CLI (systemd-report)
which
lists the metrics, and integration tests.
**System wide metrics:**
- units_by_type_total
- units_by_state_total
**Two per unit metrics:**
- unit_active_state
- unit_load_state
**A service state metric:**
- nrestarts
### Sample outputs
**units_by_type_total**:
```
{
"name" : "io.systemd.Manager.units_by_type_total",
"value" : 52,
"fields" : {
"type" : "target"
}
}
{
"name" : "io.systemd.Manager.units_by_type_total",
"value" : 82,
"fields" : {
"type" : "device"
}
}
{
"name" : "io.systemd.Manager.units_by_type_total",
"value" : 2,
"fields" : {
"type" : "automount"
}
}
```
**units_by_state_total**:
```
{
"name" : "io.systemd.Manager.units_by_state_total",
"value" : 216,
"fields" : {
"state" : "active"
}
}
{
"name" : "io.systemd.Manager.units_by_state_total",
"value" : 0,
"fields" : {
"state" : "reloading"
}
}
{
"name" : "io.systemd.Manager.units_by_state_total",
"value" : 120,
"fields" : {
"state" : "inactive"
}
}
```
**unit_active_state**:
```
{
"name" : "io.systemd.Manager.unit_active_state",
"object" : "multi-user.target",
"value" : "active"
}
{
"name" : "io.systemd.Manager.unit_active_state",
"object" : "systemd-sysusers.service",
"value" : "inactive"
}
```
**unit_load_state**:
```
{
"name" : "io.systemd.Manager.unit_load_state",
"object" : "multi-user.target",
"value" : "loaded"
}
```
**nrestarts**:
```
{
"name" : "io.systemd.Manager.nrestarts",
"object" : "user@0.service",
"value" : 0
}
{
"name" : "io.systemd.Manager.nrestarts",
"object" : "user-runtime-dir@0.service",
"value" : 0
}
```
Add JSON_BUILD_PAIR_IN_ADDR_WITH_STRING, JSON_BUILD_PAIR_IN6_ADDR_WITH_STRING,
and JSON_BUILD_PAIR_IN4_ADDR_WITH_STRING macros (plus _NON_NULL variants) that
emit both the binary address array and a human-readable string representation.
For example, an address field now emits both:
"Address": [192, 168, 1, 1]
"AddressString": "192.168.1.1"
This improves usability of the JSON/Varlink output for debugging and tooling
that consumes networkd state.
Updated networkd-json.c to use these new macros for addresses, neighbors,
nexthops, routes, routing policy rules, DNS servers, NTP servers, SIP servers,
domains, DHCPv6 prefixes, and link-local addresses.
Also updated the Varlink interface definitions to include the new string fields.
When the extensions for the final system are already set up from the
initrd we should avoid disrupting the boot process with the remount
(which currently isn't atomic) and the daemon reload for
systemd-confext and systemd-sysext. Similarly, when sysupdate ran and
updated extensions it's best to avoid the remount and daemon reload if
no changes are found.
To do this, encode the current extension state in more detail than
before where only the names of the extensions where encoded in the
overlay mount. This can also be used to provide more details about the
extension origin in "systemd-sysext status (--json=)". During the
refresh add a check whether the old state matches the new state and in
this case skip the refresh unless the user provides a flag to always
refresh. Besides the extension name and the resolved path the best
method for identification is the verity hash but that is not available
for plain image files or directories. Therefore, also include data to
check for file/directory replacements. The creation/modification times
are not always real on reproducible images or extracted archive content.
The file handle together with the unique mount ID is the next best
identifier we can use when we have no verity hash. Fall back to an inode
when we get no handle. With the creation/modification time and the path
this should be good enough. Using a unique mount ID is important (with
a fallback to the regular non-unique mount ID) instead of st_dev because
st_dev gets reused too easily, e.g., by a loop device mount and the
mount ID helps to catch this. For the mount ID to be valid it has to be
resolved before we enter the new mount namespace. Thus, it gets provided
by the image dissect logic and handed over to the sysext subprocess
which runs in a new mount namespace.
Luckily, we can rule out online modification of directories or image
files because this is anyway not well supported with overlay mounts, so
we don't do a file checksum nor do we recurse into a directory to look
for the most recently touched files. But, as said, with the
always-refresh flag one can force a reload.
Currently, `resolvectl status` (and other status commands like
`resolvectl dns $iface`) uses varlink when asked to print JSON, but uses
dbus otherwise. This PR re-implements the regular status output to use
varlink instead of dbus.
This both simplifies the `resolvectl status` code, and is an incremental
step in replacing dbus usage with varlink throughout `resolvectl`.
The first several commits move `dns-configuration.[ch]` from
`src/network/wait-online` to `src/shared`, and complete the JSON
dispatching so that all fields are available in a concrete type. The
next few commits re-implement global, link, and delegate status output.
Finally, the last commits consolidate and simplify the status logic
since varlink and the `DNSConfiguration` type can be used in all cases.
When the extensions for the final system are already set up from the
initrd we should avoid disrupting the boot process with the remount
(which currently isn't atomic) and the daemon reload for
systemd-confext and systemd-sysext. Similarly, when sysupdate ran and
updated extensions it's best to avoid the remount and daemon reload if
no changes are found.
To do this, encode the current extension state in more detail than
before where only the names of the extensions where encoded in the
overlay mount. This can also be used to provide more details about the
extension origin in "systemd-sysext status (--json=)". During the
refresh add a check whether the old state matches the new state and in
this case skip the refresh unless the user provides a flag to always
refresh. Besides the extension name and the resolved path the best
method for identification is the verity hash but that is not available
for plain image files or directories. Therefore, also include data to
check for file/directory replacements. The creation/modification times
are not always real on reproducible images or extracted archive content.
The file handle together with the unique mount ID is the next best
identifier we can use when we have no verity hash. Fall back to an inode
when we get no handle. With the creation/modification time and the path
this should be good enough. Using a unique mount ID is important (with
a fallback to the regular non-unique mount ID) instead of st_dev because
st_dev gets reused too easily, e.g., by a loop device mount and the
mount ID helps to catch this. For the mount ID to be valid it has to be
resolved before we enter the new mount namespace. Thus, it gets provided
by the image dissect logic and handed over to the sysext subprocess
which runs in a new mount namespace.
Luckily, we can rule out online modification of directories or image
files because this is anyway not well supported with overlay mounts, so
we don't do a file checksum nor do we recurse into a directory to look
for the most recently touched files. But, as said, with the
always-refresh flag one can force a reload.
This only has a small effect, which mostly matters for generating status
output. It ensures that the order of servers and domains displayed
matches the order they are stored on the daemon side, which keeps the
new resolvectl output consistent with the old.
This is one of several commits to dispatch all fields of the
DNSConfiguration JSON.
Introduce a DNSScope type, and all required JSON dispatching helpers.
The varlink API is currently missing the "DNSSEC supported" field, which
is required for to re-implement the existing resolvectl status output
with varlink instead of dbus.
Add this field to DNSConfiguration.
There's no reason to refuse copying out capsule@abc.service even if
capsule@.service already exists on the host, so let's optionally not
search for templates if we can't find an instance when looking up
units and use it in portabled.
For many use cases DynamicUser= just doesn't cut it. Specifically lots
of stuff wants to run as root. Let's support this use case by relaxing
our checks for capsules a bit.
When plymouth is enabled, the screen gets all garbled when firstboot
runs and asks questions. Disable the splash screen, but only when
actually asking questions.
unit_file_exists() currently does not follow symlinks, as it is
mainly used to test whether the unit (of any install state)
has been seen under search dirs. However, in ssh-generator
we should search for and honor whatever the distro ships,
which might be an alias or linked externally.
This commit introduces the shared code for the metrics API framework:
- Metrics API definitions
- Code to set up the varlink server
- The describe method which shows all the metrics families
- The list method which lists all the metrics
- Type definitions related to MetricFamily
- Common code to build json objects
This commit extracts the creation of the snapshot for
nspawn into a shared helper. Its not a lot of code but its
subtle so having a single place seems beneficial.
While extracting the snapshot code got tweaked/simplified based
on the feedback from Yu (many thanks!).
With that we can simplify the snapshot delete and error handling
as well. The shared helper is then going to be used by vmspawn
when it also gets the --ephemeral option.
Unlike other settings that takes multiple values, the setting takes
colon separated list of paths, but when specified as a DBus property,
it previously accepted space separated list of paths.
Let's also make the DBus property accepts colon separated lists.
Fixes#40513.
This is needed to set up extension images from the initrd with
`systemd-sysext --root=/sysroot/ merge`.
- vpick: Don't use openat directly but resolve symlinks in given root
With systemd-sysext --root= all symlinks should be followed relative to
the given root and direct openat usage doesn't work.
Remove the openat call and let pin_choice do the work with the chase
helper function to resolve the symlink in the given root.
- discover-image: Follow symlinks in a given root
So far systemd-sysext with --root= specified didn't follow extension
symlinks (such as the "current" symlinks managed by systemd-sysupdate).
The main use case is running systemd-sysext --root=/sysroot for setting
up the overlay mounts already from the initrd.
Resolve symlinks correctly but don't defend against later symlink races
that would access a path outside of the given root. Malicous live
modifications are not a realistic threat model and anyway for that one
would need to rework how the image entry is passed over up to the point
when the loop device is set up. This change here does not introduce this
weakness nor does it expose it more than before. Thus, make it explicit
that setting up the extensions for a given --root= implies a certain
trust into this given root tree that it does not try do race conditions
with symlinks to trick systemd-sysext to mount a file outside --root=.
Without a strict --image-policy= set we would anyway mount filesystems
right away which is another attack vector but, again, the main use case
is to do this for the final system which is trusted at this stage.
- sysext: Use correct image name for extension release checks
For the extension release check the image name is needed and was derived
from the backing file of the loop device. However, this can have a
different name when symlinks were resolved. The surprising behavior was
that it worked when the target name started with the extension name and
_ because that's what's supported to chop off version suffixes. However,
we should not have such strict requirements for the target name and also
allow - as version separator and entirely different names/prefixes, the
same way as we also do for directories instead of raw images.
Do not use the image name derived from the backing file of the loop
device but directly the extension name we have at hand.
- test: Add tests for handling symlinks with systemd-sysext
When we now allow following symlinks inside a --root= we should also
test that it works in various cases from simple relative and absolute
symlinks to .v being a symlink itself or its contents, both for
directory and for .raw image extensions. While at it, also add a simple
test for .v without symlinks which wasn't there for direct usage of
systemd-sysext.
get_block_device_harder_fd() currently only traverses one level of
device mapper stacking when looking for the underlying block device.
This causes issues with nested DM setups like dm-crypt on top of
dm-integrity, where we don't traverse enough to get the actual physical
device.
Fix this by iterating through all DM layers until we reach a device with
no underlying device. And while we're at it also make cgroups use the
same logic.
Fixes: #40419
The immediate impulse for this change is the fedora scriptlet which called:
/usr/lib/systemd/systemd-update-helper install-system-units cryptsetup-pre.target cryptsetup.target getty@.service ... system-systemd\x2dcryptsetup.slice system-systemd\x2dveritysetup.slice ...
which called
systemctl preset cryptsetup-pre.target cryptsetup.target getty@.service ... system-systemd\x2dcryptsetup.slice system-systemd\x2dveritysetup.slice ...
which threw an error that system-systemdx2dcryptsetup.slice does not exist
and did nothing at all. (The backslash is consumed by the shell.)
The obvious fix here is to figure out more levels of escaping… But we should
do something more robust in such cases.
If we fail in processing of a single unit, let preset all continue processing
units, report the failure through 'changes'. At the end, return failure. In
general, for operations which operate on a list of units specified by the user,
fail the whole operation if any of the individual operations failed. The only
operation where we don't do this is 'preset-all'.
$ SYSTEMD_LOG_LEVEL=debug build/systemctl --root=/ preset asdf1.servie asdf2.path; echo $?
Cannot find unit asdf1.servie.service.
Cannot find unit asdf2.path.
Failed to preset unit: Unit asdf1.servie.service does not exist
Failed to preset unit: Unit asdf2.path does not exist
1
While at it, fix double logging in the manager: dump_unit_changes() already
logs about errors, so the manager should only log on success.
This partially reverts a4f0e0da35. The
intent was good, we gather the errors, but we have no mechanism to
propagate the result, so the gathered result was ignored. In 'changes'
we can only report errors for specific units. If reading of the
directory fails, we might just as well report the error immediately.
This isn't great, but it's better then ignoring the errors. In practice,
failing halfway in this manner is unlikely, since it'd mean that the fs
is corrupted or something like that. We might as well return immediately
on such catastrophic errors.
archive_read_header(3) states that `archive_read_next_header()` returns
`ARCHIVE_WARN` if it succeeds but a non-critical error is found, so issue a
warning instead of stopping the untar process in this case.
For the extension release check the image name is needed and was derived
from the backing file of the loop device. However, this can have a
different name when symlinks were resolved. The surprising behavior was
that it worked when the target name started with the extension name and
_ because that's what's supported to chop off version suffixes. However,
we should not have such strict requirements for the target name and also
allow - as version separator and entirely different names/prefixes, the
same way as we also do for directories instead of raw images.
Do not use the image name derived from the backing file of the loop
device but directly the extension name we have at hand.
So far systemd-sysext with --root= specified didn't follow extension
symlinks (such as the "current" symlinks managed by systemd-sysupdate).
The main use case is running systemd-sysext --root=/sysroot for setting
up the overlay mounts already from the initrd.
Resolve symlinks correctly but don't defend against later symlink races
that would access a path outside of the given root. Malicous live
modifications are not a realistic threat model and anyway for that one
would need to rework how the image entry is passed over up to the point
when the loop device is set up. This change here does not introduce this
weakness nor does it expose it more than before. Thus, make it explicit
that setting up the extensions for a given --root= implies a certain
trust into this given root tree that it does not try do race conditions
with symlinks to trick systemd-sysext to mount a file outside --root=.
Without a strict --image-policy= set we would anyway mount filesystems
right away which is another attack vector but, again, the main use case
is to do this for the final system which is trusted at this stage.
Yu Watanabe
Lennart Poettering
Mike Yuan
Zbigniew Jędrzejewski-Szmek
gvenugo3
Lennart Poettering
Kai Lueke
Nick Rosbrook
Daan De Meyer
Luca Boccassi
Yaping Li
Antonio Alvarez Feijoo
Michael Vogt