Many of our services are nowadays implemented via socket activation, and
hence require sockets.target to be active to be accessible. One of them
is mute-console.socket, which we typically want to use from
systemd-firstboot.service, systemd-sysinstall.service and other related
services. Hence let's pull in basic.target rather than sysinit.target
from system-install.target since it pulls sockets.target in too.
Effectively, this doesn't change much except for pulling in a bunch more
sockets, and frankly going for sysinit.target was really a bug to begin
width.
We removed the concept of facts, so we need to update the existing
io.systemd.Basic facts provider to metrics. This commit does just
that. Its mostly mechanical.
This also means that facts.{c,h} and varlink-io.systemd.Facts.{c,h}
are gone now.
Second StorageProvider implementation, exposing regular files and
directories from a backing filesystem. In system mode the backing
directory is /var/lib/storage/, in user mode $XDG_STATE_HOME/storage/;
entries with a .volume suffix are exposed, with the inode type
determining whether the volume is reported as reg, dir or (via
symlinked/bind-mounted device node) blk.
Unlike the block provider, this one supports creating volumes
on-demand from a small set of built-in templates: sparse-file,
allocated-file, directory and subvolume.
First implementation of io.systemd.StorageProvider, exposing all block
devices known to udev (disks, partitions, dm nodes, …) as volumes of
type "blk". Names are picked from stable /dev/mapper and /dev/disk/by-*
symlinks; content-derived identifiers (by-uuid, by-label, …) are
intentionally avoided for security. Volume creation is not supported by
this backend.
Socket-activated via /run/systemd/io.systemd.StorageProvider/block.
Also adds shared storage-util.[ch] (VolumeType / CreateMode helpers)
that subsequent providers reuse.
Without this, the socket is available well before systemd-networkd.service
is able to start, because of its own After=network-pre.target ordering.
Then, if resolved handles queries before network-pre.target, it will
hang waiting for networkd to reply to hook queries.
This is currently happening in the wild with cloud-init.
The shutdown interface is currently only exposed via dbus. This PR
adds a comparable varlink implementation. It is inspired by the existing
dbus methods and implements PowerOff, Reboot, Halt, Kexec, SoftReboot
as varlink methods on a new io.systemd.Shutdown interface.
It is (intentional) simpler than the dbus for now, i.e. no Can* yet,
mostly
because I want to get feedback first (happy to do that in a followup).
The only real difficulty here is what to do about
verify_shutdown_creds()
as this is needed by both dbus and varlink and its dbus only. I went for
an ugly but (hopefully) pragmatic choice (see the commit message for
details). But I can totally understand if a refactor instead is
preferred.
Add CpuUsage, MemoryUsage, IOReadBytes, IOReadOperations, and
TasksCurrent in a standalone socket-activated varlink service. These
metrics are gathered from the kernel via cgroup files and PID1's only
role is mapping unit names to cgroup paths — a separate process can
query PID1 once for that mapping and then read the cgroup files
directly, minimizing PID1 involvement.
The new systemd-report-cgroup-metrics service listens at
/run/systemd/report/io.systemd.CGroup and exposes:
- io.systemd.CGroup.CpuUsage
- io.systemd.CGroup.IOReadBytes
- io.systemd.CGroup.IOReadOperations
- io.systemd.CGroup.MemoryUsage (with type=current/available/peak)
- io.systemd.CGroup.TasksCurrent
This is spun out of #41078 and based on top of it. Will rebase once
that's merged.
Add CpuUsage, MemoryUsage, IOReadBytes, IOReadOperations, and
TasksCurrent in a standalone socket-activated varlink service.
The new systemd-report-cgroup service listens at
/run/systemd/report/io.systemd.CGroup and exposes:
- io.systemd.CGroup.CpuUsage
- io.systemd.CGroup.IOReadBytes
- io.systemd.CGroup.IOReadOperations
- io.systemd.CGroup.MemoryUsage (with type=current/available/peak)
- io.systemd.CGroup.TasksCurrent
The shutdown interface is currently only exposed via dbus. This commit
adds a comparable varlink implementation. It is inspired by the existing
dbus methods and implements PowerOff, Reboot, Halt, Kexec, SoftReboot.
It is (intentional) simpler than the dbus for now, i.e. strictly root
only. To match dbus we will need to the functionality
of verify_shutdown_creds() which is dbus-ish right now and would
need some refactor.
For the same reason it does not do the Can* methods - we will need
the verify_shutdown_creds() equivalent first.
While it is possible to disable sysext/confext merging in the main system
with 'systemctl disable', sysext/confext are always merged in the initrd,
both by systemd-{sys,conf}ext-initrd.service and by
systemd-{sys,conf}ext-sysroot.service and especially the latter can be
unexpected. Provide kernel cmdline options systemd.{sys,conf}ext=0 and
rd.systemd.{sys,conf}ext=0 covering all options.
The new services are used to activate system and configuration extensions
for the main system from the initrd, this allows to overcome the limitation
that sysext/confext cannot be used to update the resources which are required
in the earliest boot of the system (before systemd-sysext/systemd-confext
start).
The collection of facts is entirely unprivileged and has very little to
do with PID1. PID1 is privileged and single-threaded and a point of
contention, so we shouldn't put things in PID1 that don't need to be
there. A separate service can be enabled/disabled/started/stopped at
will, is easy to sandbox, etc. If it turns out to be necessary to
collect some facts through PID1 in the future, we can always add a
smaller facts endpoint to PID1 again.
This makes sure we don't use the "hardware" or "verity" nvpcrs before
the NV anchor measurement is done.
This is mostly to avoid confusing output, and to indirectly ensure the
nvpcr allocation in tpm2-setup is the load bearing one, but it should
not be load bearing for security afaics.
This has been requested previously for PCR 7 (#40567), but let's do that
for all firmware owned PCRs, since some firmwares forget to measure
their own separator. Let's hence measure our own guranteed one.
Fixes: #40567
So far we always conditioned our TPM magic on the UKI having detected
TPM support in the firmware. This is a bit limiting when we want to
support a software TPM that is not visible to the firmware. Hence let's
split this up, and add a separate control that can be set via the kernel
command line. However, as before, let's by default inherit the firmare
TPM discovery state into it, to retain the current behaviour unless
overriden.
With this in place, boot with "systemd.tpm2_measured_os=1
systemd.tpm2_software_fallback=1" on the kernel cmdline to get the swtpm
fallback and then a measured OS based on it.
For TPM-less systems it's sometimes valuable to have a fill-in software
TPM running from early boot on, so that TPM-based functionality can
"just work" and rely on TPM semantics, even if it's at a substantially
weaker security level.
This adds a wrapper around swtpm. It's a binary that chainloads swtpm
but does a few preparatory steps and integrates into systemd's logic
otherwise.
All this is then exposed as systemd-tpm2-swtpm.service.
The service is not hooked into much yet, that is added in later commits.
This is a client tool to the systemd-imdsd@.service added in the
previous commit. It's mostly just a 1:1 IPC client via Varlink. It can
be used to query any IMDS key, but it's primary usecase is to acquire
the "userdata" from IMDS. Moreover, if invoked with the --import switch
it will check if the userdata contains a list of system credentials. If
so, it will import them into the local credstore. If the userdata does
not look like a list of system credentials no operation is executed,
under the assumption the data is intended for cloud-init instead.
It also imports a couple of other fields, if available and recogniuzed,
such as SSH keys and the hostname.
This service's job is to talk to a VM associated IMDS service provided
by the local Cloud. It tries to abstract the protocol differences
various IMDS implementations implement, but does *not* really try to
abstract more than a few basic fields of the actual IMDS metadata.
IMDS access is wrapped in a Varlink API that local clients can talk to.
If possible this makes use of the IMDS endpoint information that has
been added to hwdb in the preceeding commit. However, endpoint info can
also be provided via kernel command line and credentials. For debugging
purposes we also accept them via environment variables and command line
arguments.
This adds a concept of early-boot networking, just enough to be able to
talk to the IMDS service. It is minimally configurable via a kernel
cmdline option (and a build-time option): the user may choose between
"locked" and "unlocked" mode. In the former mode direct access to IMDS via
HTTPS is blocked via a prohibit route (and thus all IMDS communication
has to be done via systemd-imdsd@.service). In the latter case no such
lockdown takes place, and IMDS may be acquired both via this new service
and directly. The latter is typically a good idea for compatibility with
current systems, the former is preferable for secure installations.
Access to IMDS fields is controlled via PK.
CAP_SYS_PTRACE for making sure we can open mount namespaces of
peers via /proc/<pid>/ns and CAP_SYS_CHROOT for making sure we can
join those mount namespaces.
Currently the varlink interface for hostname is only available
after sysinit. This means it is not available until systemd-firstboot
is finished. But there is information like the boot-id in there that
is useful to get early.
My use-case is to query the system early via the varlink-http-bridge
and currently I can't get data from io.systemd.Hostname until
systemd-firstboot is completed which is a bit limiting.
So to fix it this commit sets DefaultDependencies=no on both the socket
and service units.
It also changes hostnamed.c to use
bus_open_system_watch_bind_with_description() which means we will
reconnect once dbus is available. This mimics what resolved-bus.c
is doing (and which was originally introduced in d7afd945b).
Thanks to Lennart for pointing this out.
Let's bubble up failures all the way until they reach the services, but
then let's carefully gracefully handle some of them, that are about
issues not immediately actionable to the admin, even if they are
potentially quite problematic.
We expose this via --private-users-delegate= which takes the number of
ranges to delegate. On top of delegating the ranges, we also mount in
the nsresourced socket and the mountfsd socket so that nested containers
can use nsresourced to allocate from the delegated ranges and mountfsd to
mount images.
Finally, we also create /run/systemd/dissect-root with systemd-tmpfiles to
make sure it is always available as unpriv users won't be able to create it
themselves.
mkosi does all of its environment setup in an unprivileged user
namespace with an identity mapping. When it invokes nspawn and nspawn
tries to get a transient userns from nsresourced, this fails as no
transient ranges are mapped into mkosi's unprivileged userns (as doing
so would require privileges).
To fix this problem, let's allow allocating unprivileged self user
namespaces in nsresourced, similar to what the kernel allows, except that
we also support delegations for these. This means that mkosi can get its
unprivileged userns as before from nsresourced, but it can also request a
delegated 64K range inside that userns as well, which nsresourced can then
allocate to nspawn later when it asks for one.
Similar to the kernel, we disallow setgroups for self mappings. However,
instead of doing this via /proc/self/setgroups, which applies to the current
user namespace and all its child user namespaces, we use the BPF LSM to deny
setgroups instead, so that it can still be allowed for child user namespaces.
We need this because as soon as a container launches in a child user namespace
using one of the delegated mappings, it has to be able to do setgroups() to be
able to function properly.
To allow mapping the root user, we need to add the CAP_SETFCAP capability to
nsresourced.
The socket failed on boot due to a ordering problem caused by
systemd-sysusers running later and thus the socket group was still
missing. The socket unit can only start when systemd-sysusers created
the group.
Order the socket after systemd-sysusers.service to have the group
available.
For AF_UNIX sockets connection sources are accounted for
based on UID, hence in user scope this effectively
limits total number of connections, which is not really
desirable.
Follow-up for a109189fab
This follows the existing practice for
systemd-journal-{upload,gatewayd}.service,
as I think allocating a full-blown user
specifically for this purpose is an overkill.
And with DynamicUser=yes we can also take
advantage of implied sandboxing.
We already have some varlink support for the journal to perform
some actions like `Rotate`. It would be nice to be able to query
the journal via varlink too so this commit adds a new varlinkctl
based journal service that exposes a single GetEntries() call
to retrieve journal entries. Basic filtering is supported and
we can expand the API as needed.
This is a separate `io.systemd.JournalControl` [1] service from the
existing `io.systemd.Journald` to decouple read and write (thanks
to Lennart for suggesting this).
This also extracts some shared helper so that we do not duplicate
code when generating the json or when adding the filters.
[1] The name mirrors the bootctl->io.systemd.BootControl naming.
So far we didn't provide any concept to enumerate local Varlink
services. Let's change that.
Let's define very light-weight scheme for this: provide a well-known dir
/run/varlink/registry/ where services that implement public interfaces
can link their sockets into. When enumerating services it's thus
sufficient to enumerate inodes in that directory.
The usecase for this is twofold:
1. It's simply very useful to be able to see which public services are
bound on the local system, for debugging/admin/development purposes.
2. At Amutable we'd like to optionally provide a HTTP-to-Varlink bridge
on individual nodes, that allows remote peers (after authentication)
to access local Varlink services. For that it's essential we know the
list of services and their entrypoints to expose, it would be
security-wise highly problematic for clients to provide AF_UNIX
entrypoint paths when connecting. hence: let's instead just have a
dir with the public stuff, and let's ensure the HTTP-to-Varlink
bridge simply exposes that stuff, and nothing else.
Non-public interfaces (such as the oomd interfaces between PID 1 and
oomd), and interfaces with multiple implementors (such as the resolved
hook interface, or the metrics collection stuff) should not be linked
in.
This is inspired by the Varlink.org "registry" concept, briefly
explained here:
https://varlink.org/FAQ#how-do-i-find-the-service-which-implements-a-local-interface
Note however that the described Varlink interface is not actually
implemented here, the directory is introduced however in a fashion that
conceptually matches the registry defined there, and would allow us to
implement the registry interface on top of it. (One of the reason the
registry Varlink API is not implemented right now is that the URI format
it relies on is entirely unspecified in the Varlink docs right now. Some
research needs to be done to extract what's implemented in the reference
implementation and to determine how it maps to the Varlink entrypoint
address format systemd's own tooling currently uses)
This primarily installs the symlinks via Symlinks= in unit files and via
a new tmpfiles.d/ drop-in. But since we touch all .socket units relating
to Varlink this also sets the FileDescriptorName= to varlink for each,
just to minimize diffrences and make things work more alike (the
services in questin don't care about the name, so this doesn't change).
In one case we replace a pair of separate sockets for two closely
related varlink services by a socket and a symlink, so that we can
safely use Symlinks= to also install the registry symlinks.
agetty from util-linux is meanwhile following the configuration file
specification for /etc/issue. The usage of "--issue-file" breaks this
on distributions with current util-linux.
This partially reverts commit e58ba80a40. Those
aliases were removed as part of the removal of sysvinit compat, but the removal
was immediately reported as a regression by users. In particular, users can
have one of those set as the default target, or use those names in Wants= or
Requires= dependencies. At least in Fedora, the removal was immediately
reverted.
When looking at our "portability and stability promise", I noticed that those
units are explicitly listed in the promise. Let's restore them upstream. They
are conditionalized under a new -Dcompat-sysv-interfaces=bool option, so that
downstreams like Arch that wasnt to strictly avoid legacy interfaces can easily
disable them. They are dropped from the docs, like we do for all deprecated
interfaces.
In Fedora, kmsconvt@.service is starting to be used instead of getty@.service
to have nicer font handling. This means that we need the autovt@.service alias
point to the new unit. So far the alias was done through a static symlink
because there was little reason to change it. Let's use [Install] instead so
the decision which implementation to use can be made after installation.
https://bugzilla.redhat.com/show_bug.cgi?id=2429626https://fedoraproject.org/wiki/Changes/UseKmsconVTConsole
This does not yet support directory images properly
as systemd itself does not support unpriv directory
images properly yet.
The user profiles are a copy of the system profiles but without
DynamicUser=yes (can't be used by user managers) and without
ProtectHome=yes (this masks /home which breaks StateDirectory= which
is lcoated inside /home)
Zbigniew Jędrzejewski-Szmek
Lennart Poettering
Michael Vogt
Lennart Poettering
Nick Rosbrook
Yaping Li
Vitaly Kuznetsov
Hadi Chokr
Daan De Meyer
Daan De Meyer
Kai Lüke
Mike Yuan
Yu Watanabe
nikstur
Thorsten Kukuk
Zbigniew Jędrzejewski-Szmek