This is a series of commits which adds a feature needed by GNOME OS'
installer. This was show during All Systems Go 2025 talk:
https://cfp.all-systems-go.io/all-systems-go-2025/talk/QRJVL3/
To sum up this PR, this changes first systemd-repart to use BLKPG
partition instead of loop devices when possible. We need then to always
rescan the partitions to try remove partitions if it failed. We allow
encrypted partitions to stay activated and with a chosen name. And we
add a new partition configuration `BtrfsReplace=`.
Note that "replace" comes from the command `btrfs replace`. But in the
case of systemd-repart, maybe "inplace" or "move" would make more sense.
I open to suggestions.
If it is better I can split this into several PRs.
The commits:
## repart: Reuse the backing fd for fdisk
Because fdisk_assign_device tries to open block devices with O_EXCL,
when it does it blocks cryptsetup from using partition block devices for
the same disk.
Since we already have a file descriptor for the device, we can just
share it and use fdisk_assign_device_by_fd instead.
## repart: Use blkpg partitions instead of loop devices when possible
We will want to allow future features to keep some devices mounted or
active. So in order to avoid leaving a mess of many loop devices, we can
just already use the partition block device already.
## repart: Rescan disk on failure if we create blkpg partitions on the
fly
Since we did not write the partition table, then the created partitions
should get removed on error.
## repart: Allow keeping luks2 volumes opened
## repart: Add BtrfsReplace=
BtrfsReplace=/mntpnt will move the btrfs filesystem from mount point to
the partition created. After moving, it will resize to take the whole
partition.
This is useful for OS installers that move a live system into a disk and
do not require a reboot.
## repart: Add VolumeName=
When a luks2 device mapper is to be kept alive after execution of
systemd-cryptsetup, the name of the volume will be taken from this
value.
## test: Add test for repart's BtrfsReplace
$PATH might point to a systemd checkout containing
a newer version of systemd-journal-remote which we
should use, hence prefer an executable from $PATH
over the one from /usr/lib/systemd.
BlockDeviceReplace=/mntpnt will move the btrfs filesystem from mount point to
the partition created. After moving, it will resize to take the whole
partition.
This is useful for OS installers that move a live system into a disk and
do not require a reboot.
Because fdisk_assign_device tries to open block devices with O_EXCL, when it
does it blocks cryptsetup from using partition block devices for the same
disk.
Since we already have a file descriptor for the device, we can just share it
and use fdisk_assign_device_by_fd instead.
This requires at least libfdisk 2.35 (part of util-linux) which was
released in 2020.
Without CAP_SYS_ADMIN, btrfs_ioctl_snap_destroy() runs an
inode_permission(MAY_WRITE) check against the target subvolume root, which
btrfs_permission() rejects with EROFS for a read-only subvolume. As a
result, unprivileged removal of a read-only subvolume fails — both via
btrfs_subvol_remove_at() directly and via the recursive cleanup path used
by rm_rf_subvolume_and_freep(), which propagates the EROFS up.
Detect EROFS after the destroy ioctl, clear the RDONLY flag (only inode
ownership is required for BTRFS_IOC_SUBVOL_SETFLAGS), and retry once.
While at it, fix the surrounding comments: BTRFS_IOC_SNAP_DESTROY drops the
entire subvolume tree, so regular files inside are irrelevant; ENOTEMPTY
from the ioctl indicates nested subvolumes (BTRFS_ROOT_REF_KEY entries) via
may_destroy_subvol(), not non-empty contents.
We have different help strings for --user/--system in different places, so this
only covers a subset of --system/--user instances. But this particular help
seems to be the most widely used.
(In a few cases, the help string is fixed: it should be "system mode", not
"per-system mode".)
BTRFS_IOC_SEARCH is only available to root in the
initial userns. This means we fail to recursively
snapshot even if a subvolume has no nested subvolumes
at the moment.
Let's fix this by using the newer btrfs ioctls which
do work even if we don't have CAP_SYS_ADMIN in the initial
userns.
This makes sure that whenever we want to show the OS name we can show
the fancy name. Thus this moves the escaping/validation of the fancy
name out of hostnamed into generic code, and then makes use of it in
sysinstall,firstboot,prompt-util.
UEFI firmware can report the currently-active keyboard layout via
EFI_HII_DATABASE_PROTOCOL.GetKeyboardLayout(). The layout descriptor
includes an RFC 4646 / BCP 47 language tag (e.g. "en-US"). Query this
from sd-boot/sd-stub and write it to a new LoaderKeyboardLayout EFI
variable, advertised through a new EFI_LOADER_FEATURE_KEYBOARD_LAYOUT
feature bit.
On the OS side, systemd-vconsole-setup reads the variable as a
lowest-priority fallback for the console keymap. To map the BCP 47
tag to a vconsole keymap we extend /usr/share/systemd/kbd-model-map
with an optional sixth column listing the comma-separated BCP 47 tags
each row covers; a new find_vconsole_keymap_for_bcp47() helper walks
the file, preferring an exact tag match and otherwise falling back to
the row whose tag matches the input's primary subtag. Credentials,
/etc/vconsole.conf, and vconsole.keymap= on the kernel command line
continue to take precedence.
bootctl status surfaces the new variable, printing the language tag
or "n/a (not reported by firmware)" when sd-boot advertises the
feature but the firmware HII database didn't expose a layout (common
on QEMU without a USB keyboard, since EDK2's PS/2 driver does not
register an HII keyboard layout).
The Varlink iterface said the definitions directory was mandatory, and
so did the dispatch table. But that's nonsense, the code is completely
fine to operate without (same as cmdline repart invocations): it will
just use the standard definitions dir.
Fix that.
Add varlink context and runtime builders for .timer units:
TimerContext: MonotonicTimers (with MonotonicTimerBase enum), CalendarTimers, Unit, OnClockChange, OnTimezoneChange, AccuracyUSec, RandomizedDelayUSec, RandomizedOffsetUSec, FixedRandomDelay, Persistent, WakeSystem, RemainAfterElapse, DeferReactivation
TimerRuntime: Result (TimerResult enum), NextElapseUSecRealtime, NextElapseUSecMonotonic, LastTriggerUSec
MonotonicTimerSpec and CalendarTimerSpec are separate types since they have different value types (int vs string). MonotonicTimerBase and TimerResult are proper varlink enum types.
Compared to the old io-systemd-Unit-List branch, this adds RandomizedOffsetUSec and DeferReactivation (both present in D-Bus but previously missing), and adds full runtime fields.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Add varlink context and runtime builders for .swap units:
SwapContext: What, Priority, Options, TimeoutUSec, ExecActivate, ExecDeactivate
SwapRuntime: ControlPID, Result, CleanResult, UID, GID (SwapResult enum)
SwapResult is exposed as a proper varlink enum type. Runtime follows the same pattern as MountRuntime (ControlPID, Result, CleanResult, ref UID/GID).
The integration test is conditional since swap units may not be present on all systems.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Add varlink context and runtime builders for .scope units:
ScopeContext: OOMPolicy (enum), RuntimeMaxUSec, RuntimeRandomizedExtraUSec, TimeoutStopUSec
ScopeRuntime: Result (ScopeResult enum)
Both OOMPolicy and ScopeResult are exposed as proper varlink enum types.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
OOMPolicy is used by both io.systemd.Manager (DefaultOOMPolicy) and io.systemd.Unit (ScopeContext.OOMPolicy), so it belongs in the shared common types alongside ManagedOOMMode and EmergencyAction.
Add varlink context and runtime builders for .path units:
PathContext: Paths (array of PathType enum + path specs), Unit, MakeDirectory, DirectoryMode, TriggerLimit
PathRuntime: Result (PathResult enum)
Both PathType and PathResult are exposed as proper varlink enum types.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
PR #41583 (io.systemd.Unit.StartTransient) introduced several new varlink IDL enum types without corresponding enum consistency tests:
- JobType, JobState, JobResult in the new io.systemd.Job interface
- ServiceType in the Unit interface's ServiceContext
Add a new test-varlink-idl-job test file covering all three Job enums, and add ServiceType coverage to the existing test-varlink-idl-unit test. Export vl_type_ServiceType (was static) so it can be referenced from the test.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Refactor curl-util to use the same per-request, refcounted, cancellable
slot model as sd-bus, sd-varlink and qmp-client.
CurlGlue becomes opaque and refcounted, and dispatches per-slot
completion callbacks through CURLOPT_PRIVATE instead of a single
g->on_finished demux that every caller had to switch on. The new
curl_glue_perform_async(g, easy, cb, userdata, &slot) replaces
curl_glue_add + the on_finished/userdata wiring.
CurlSlot is the per-request handle: it owns the easy handle,
curl_slot_unref does curl_multi_remove_handle + curl_easy_cleanup
(which doubles as cancel since remove aborts in-flight transfers
without queuing CURLMSG_DONE), and floating slots (ret_slot=NULL) are
kept alive in the glue's slot set until the callback fires. Drop the
userdata parameter from curl_glue_make: CURLOPT_PRIVATE is now used
internally to route completions to the slot.
Migrate pull-job and the pull-{oci,raw,tar} drivers, and imdsd, to the
new shape. PullJob.curl becomes PullJob.slot; pull_job_curl_on_finished
becomes a per-slot callback. imdsd routes its token-vs-data branch off
slot identity rather than easy-handle pointer comparison. Both daemons
drop the global on_finished/userdata wiring on the glue. pull_job_finish
and context_fail{,_full} now return int (always 0) so the callbacks
stay in the `return finish(...);` style.
Add test-curl-util covering glue lifecycle, easy-handle defaults,
floating and non-floating perform paths, cancel-via-slot-unref (verified
by a sentinel request that drives the loop to completion), and three
concurrent requests on a single glue. Tests fetch local files via
file:// URLs so no network is needed; libcurl availability is probed
once via dlopen_curl in intro().
We have a number of *_unref_and_replace macros. One could think that
they are like the various free_and_replace variants, but they actually
create a new ref to the passed object. The free_and_replace variants
take ownership of the argument. This inconsistency is surprising. Rename
all those functions to have "_new_ref" at the end to make the difference
clear.
The more(1) pager (part of util-linux) now supports secure mode,
which can be enabled with the MORESECURE environment variable.
Adding support for more(1) in secure mode serves as an alternative
for systems that do not have less installed or for users who prefer
it.
Commit d2fce960f9cac740 introduced secure mode in more(1) and the
new feature is available in version 2.42 of util-linux.
Define two new methods on the generic 'MachineInstance' Varlink
interface that systemd-vmspawn (this series) and (future)
systemd-nspawn implement on their per-machine control sockets:
AddStorage(fileDescriptorIndex, name, config?) -> ()
Attach a storage volume — the caller passes an fd previously
acquired from a StorageProvider, plus a unique name of the form
'<provider>:<volume>' that identifies this binding for later
removal, plus a backend-specific 'config' field (vmspawn: guest
device type; future nspawn: mount path).
RemoveStorage(name) -> ()
Detach a previously-added storage volume.
Plus errors NoSuchStorage, StorageExists, StorageImmutable (the volume
was attached at boot and cannot be removed), BadConfig, and
ConfigNotSupported. Names follow the io.systemd.StorageProvider
vocabulary (NoSuchVolume, BadTemplate, TypeNotSupported, etc.) so the
two interfaces are visually consistent.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
storagectl's mount.storage helper bundles "open StorageProvider socket
+ Acquire() + dispatch reply + take fd" inline. Future consumers
(systemd-vmspawn boot-time --bind-volume, machinectl bind-volume) need
the same dance.
Factor it into a single libshared helper that takes the Acquire()
parameters by value and returns the fd plus the actual type/read-only
flags. Library code, so no logging — varlink errors are surfaced via
sd_varlink_error_to_errno() and the StorageProvider error_id is
returned to the caller via reterr_error_id (caller decides how to
format messages).
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Add a universal parser for the colon-separated grammar
'PROVIDER:VOLUME[:CONFIG][:K=V,K=V,…]' that backs --bind-volume on
systemd-vmspawn (next), machinectl bind-volume, and the future nspawn
+ service-manager BindVolume= integrations.
The 'config' field is opaque to shared code and interpreted per
backend (vmspawn: a DiskType name, future nspawn: a mount path). The
trailing key=value list is parsed into the io.systemd.StorageProvider
.Acquire() parameters (template, create, read-only/ro, size/create-size
and request-as), with values validated against the existing
storage-util enums and validators. Provider/volume names are checked
with storage_provider_name_is_valid() and storage_volume_name_is_valid();
the combined "<provider>:<volume>" string is also validated as
string_is_safe so it is safe to use as a QEMU device id.
Add a test-machine-util unit test covering the happy paths plus a
handful of malformed inputs.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The storage backend providers (block, fs) and storagectl currently each
extract storage-util.c into their target. Several upcoming consumers
(machine-util's BindVolume parser, vmspawn's hotplug glue, machinectl's
new bind-volume verbs) need the StorageProvider type/string-table
helpers and a future shared Acquire client helper.
Move storage-util.{c,h} to src/shared so libshared exports the symbols
once and every consumer (storage providers, storagectl, libshared
itself) picks them up by linking libshared. Drop the now-redundant
'extract'/'objects' wiring in src/storage/meson.build.
No code changes; this is purely a relocation.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The mount.storage helper open-codes the conventional 64K UID/GID
delegation block size as 0x10000 / 0x10000U in four places. Several
other places in the tree do the same (nspawn's arg_uid_range default,
homed's mount setup, …), but with no shared name.
Add USERNS_RANGE_SIZE in user-util.h alongside UID_NOBODY and friends,
and switch storagectl over to it. Other call sites can adopt it
incrementally.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
We removed the concept of facts, so we need to update the existing
io.systemd.Basic facts provider to metrics. This commit does just
that. Its mostly mechanical.
This also means that facts.{c,h} and varlink-io.systemd.Facts.{c,h}
are gone now.
If all entries of a menu prompt start with the same prefix, let's
preselect the prefix to enhance user experience.
This is particularly relevant when prompting for a disk to install
things on, as typically they all start with the same prefix /dev/, and
if there's only a single target medium discoverable, then we can even
fill it out fully.
Here we have the unusual situation that the option list is
conditionalized. I thought about embedding some "tag" information in
individual options to allow the options to be filtered by some arbitrary
conditions. But it seems that using groups works quite well. It wouldn't
scale well if there was a lot more options and conditions, but for the
current set it's good enough.
For options that are not supported in a given service, we print a custom
message ("This service does not support [this] option"), instead of the
generic "Unknown option …". I think this is actually better: we don't
have to pretent that we don't know about the option, and can directly
say that the it's a valid option in general but this service does not
support it (yet).
This converts systemd-homed, systemd-hostnamed, systemd-importd,
systemd-localed, systemd-logind, systemd-machined, systemd-networkd,
systemd-portabled, systemd-resolved, systemd-sysupdated,
systemd-timedated, and systemd-timesyncd.
When we add introspection of the option data, we'll somehow have to deal
with conditionalization. But let's cross that bridge when we need to.
I thought that this would require some bigger changes, but it turns out
that the existing functionality is good enough with some minor
adjustments if used appropriately.
The behaviour of help_section() is changed to simplify all callers.
partition_is_luks2_integrity() was returning -EINVAL when it
encountered a non-LUKS2 header (e.g. LUKS v1), which caused the
caller to abort the entire disk dissection. A LUKS v1 partition
simply isn't LUKS2-with-integrity, so return 0 instead and let
dissection continue normally.
Daan De Meyer
Yu Watanabe
Valentin David
Ivan Kruglov
Lennart Poettering
Zbigniew Jędrzejewski-Szmek
hschloss
Ambareesh Balaji
Christian Brauner
Michael Vogt
Diego Viola
Zbigniew Jędrzejewski-Szmek
Nandakumar Raghavan