This is a helper macro that defines a function to drop elements of an
array but not the array itself. I used the "_many" suffix because it
most closely matches what happens here: we are calling the cleanup
function a bunch of times.
Uses stop_at_first_nonoption for POSIX-style option parsing.
--help output is the same, apart from whitespace differences
and common strings.
The error message for --ssh-key-type= is fixed.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Drop the --cxl= option and unconditionally enable cxl=on the QEMU
machine type whenever the host architecture supports it (x86_64 and
aarch64). The flag was only added for testing parity with mkosi's CXL=
setting and there is no reason to leave it as an opt-in toggle: with no
pxb-cxl device or cxl-fmw window attached, enabling it on the machine
only reserves a small MMIO region and emits an empty CEDT, so the cost
is negligible while removing one knob users would otherwise have to
flip explicitly to exercise the CXL code paths in QEMU.
Replace the random hex suffix in the runtime directory with the machine
name, changing the layout from /run/systemd/vmspawn.<random> to
/run/systemd/vmspawn/<machine-name>/.
This makes runtime directories machine-discoverable from the filesystem
and groups all vmspawn instances under a shared parent directory, similar
to how nspawn uses /run/systemd/nspawn/.
Use runtime_directory_generic() instead of runtime_directory() since
vmspawn is not a service with RuntimeDirectory= set and the
$RUNTIME_DIRECTORY check in the latter never succeeds. The directory is
always created by vmspawn itself and cleaned up via
rm_rf_physical_and_freep on exit. The parent vmspawn/ directory is
intentionally left behind as a shared namespace.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Bundle scope, buses, and registration success booleans into a
MachineRegistrationContext struct. This eliminates the reterr_registered_system and
reterr_registered_user output parameters from
register_machine_with_fallback_and_log() and the corresponding input
parameters from unregister_machine_with_fallback_and_log().
The struct carries state from registration to unregistration so the
caller no longer needs to manually thread individual booleans between
the two calls.
register_machine_with_fallback_and_log() goes from 7 to 3 parameters,
unregister_machine_with_fallback_and_log() goes from 5 to 2.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Replace the long positional parameter lists in register_machine() and
register_machine_with_fallback_and_log() with a MachineRegistration
struct that bundles all machine-describing fields.
This reduces register_machine() from 13 parameters to 3 and
register_machine_with_fallback_and_log() from 17 parameters to 7.
Callers now use designated initializers, which makes omitted fields
(zero/NULL/false) implicit and the code much more readable.
Field names are aligned with the existing Machine struct in machine.h
(id, root_directory, vsock_cid, ssh_address, ssh_private_key_path).
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The runtime bus (user bus in user scope, system bus in system scope) is
only needed for scope allocation (!arg_keep_unit) or machine registration
(arg_register != 0). When both are disabled the bus was still opened
unconditionally which causes unnecessary failures if the user bus is
unavailable.
Gate the runtime bus opening on the same condition nspawn already uses.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The error message incorrectly says "system bus" when the code is
actually opening the user bus via sd_bus_default_user().
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Rename nspawn's --user=NAME option to --uid=NAME for selecting the
container user. The -u short option is preserved. --user=NAME and
--user NAME are still accepted but emit a deprecation warning. A
pre-parsing step stitches the space-separated --user NAME form into
--user=NAME before getopt sees it, preserving backwards compatibility
despite --user now being an optional_argument.
Repurpose --user (without argument) and --system as standalone
switches for selecting the runtime scope (user vs system service
manager).
Replace all uses of the arg_privileged boolean with
arg_runtime_scope comparisons throughout nspawn. The default scope
is auto-detected from the effective UID.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Move register_machine() and unregister_machine() from
vmspawn-register.{c,h} into shared machine-register.{c,h} so both
nspawn and vmspawn can use the same implementation.
The unified register_machine() uses varlink first (for richer
features like SSH support and unit allocation) with a D-Bus
RegisterMachineWithNetwork fallback for older machined. The
interface adds a class parameter ("vm" or "container") and
local_ifindex for nspawn's network interface support.
The unified unregister_machine() similarly tries varlink first
(io.systemd.Machine.Unregister) before falling back to D-Bus.
Both register_machine() and unregister_machine() only log at debug
level internally, leaving error/notice logging to callers.
Add register_machine_with_fallback() which tries system and/or user
scope registration based on a RuntimeScope parameter
(_RUNTIME_SCOPE_INVALID for both), and
unregister_machine_with_fallback() as its counterpart. Both use
RET_GATHER() to collect errors from each scope.
Make --register= a tristate (yes/no/auto) defaulting to auto. When
set to auto, registration failures are logged at notice level and
ignored. When set to yes, failures are fatal.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
When a PTY is allocated for the console, QEMU's own stdio file
descriptors were still inherited directly from vmspawn, meaning any
output QEMU writes to stdout/stderr (e.g. warnings) would bypass the
PTY forwarder and go straight to the terminal. Similarly, QEMU could
read directly from the terminal's stdin.
Fix this by opening the PTY slave side and passing it as stdio_fds to
the fork call with FORK_REARRANGE_STDIO, so that all of QEMU's I/O
goes through the PTY and is properly forwarded.
Switch the negation character for firmware feature exclusion from
"!" to "~" to be consistent with other systemd options that support
negation such as SystemCallFilter=.
The machine types in QEMU firmware descriptions are glob patterns
like "pc-q35-*", so we use strstr() substring matching to check if
our machine type is covered by a given firmware entry.
Add --cxl=BOOL option to enable CXL (Compute Express Link) support in
the virtual machine. CXL is a high-speed interconnect standard that
allows CPUs to access memory attached to devices such as accelerators
and memory expanders, enabling flexible memory pooling and expansion
beyond what is physically installed on the motherboard. When enabled,
adds cxl=on to the QEMU machine configuration. Only supported on x86_64
and aarch64 architectures.
This is added for testing purposes and for feature parity with mkosi's
CXL= setting.
Extend --ram= to accept an optional maximum size for memory hotplug,
using the syntax --ram=SIZE[:MAXSIZE] (e.g. --ram=2G:8G). When a
maximum is specified, the maxmem key is added to the QEMU memory
configuration section to enable memory hotplug up to the given limit.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
The ICH9-LPC disable_s3 global QEMU config was a workaround for an
OVMF limitation where S3 resume didn't work with X64 PEI + SMM. SMM is
required for secure boot as it prevents the guest from writing directly
to the pflash, bypassing UEFI variable protections. With X64 PEI + SMM
enabled and S3 advertised, OVMF would hang on S3 resume. The
workaround was to tell QEMU not to advertise S3 support.
This limitation has been resolved in edk2 — the S3Verification() check
was removed in edk2 commit 098c5570 ("OvmfPkg/PlatformPei: drop
S3Verification()") after edk2 gained native X64 PEI + SMM + S3 resume
support. See https://github.com/tianocore/edk2/commit/098c5570.
Drop the now-unnecessary ICH9-LPC disable_s3 config entirely, and
guard the cfi.pflash01 secure=on setting with an x86 architecture
check since SMM is x86-specific and this option is invalid on ARM.
Let's add make three arguments optional, by splitting inspect_kernel()
from inspect_kernel_full().
Let's also downgrade logging to debug, so that this becomes more
library-like. Let's log on the call-site instead.
Add a --console-transport= option that selects between virtio-serial
(the default, appearing as /dev/hvc0) and a regular serial port
(appearing as /dev/ttyS0 or /dev/ttyAMA0 depending on architecture).
This is primarily useful for testing purposes, for example to test
sd-stub's automatic console= kernel command line parameter handling. It
allows verifying that the guest OS correctly handles serial console
configurations without virtio.
When serial transport is selected, -serial chardev:console is used on
the QEMU command line to connect the chardev to the platform's default
serial device. This cannot be done via the QEMU config file as on some
platforms (e.g. ARM) the serial device is a sysbus device that can only
be connected via serial_hd() which is populated by -serial.
When running in a console mode (interactive, native, or read-only),
propagate the host's $TERM into the VM by adding TERM= and
systemd.tty.term.hvc0= to the kernel command line.
TERM= is picked up by PID 1 and inherited by services on /dev/console
(such as emergency.service). systemd.tty.term.hvc0= is used by services
directly attached to /dev/hvc0 (such as serial-getty@hvc0.service) which
look up $TERM via the systemd.tty.term.<tty> kernel command line
parameter.
While systemd can auto-detect the terminal type via DCS XTGETTCAP, not
all terminal emulators implement this, so explicitly propagating $TERM
provides a more reliable experience. We skip propagation when $TERM is
unset or set to "unknown" (as is the case in GitHub Actions and some
other CI environments).
Previously this was handled by mkosi synthesizing the corresponding
kernel command line parameters externally.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Switch runtime virtiofs mount configuration from systemd.mount-extra=
kernel command line parameters to the fstab.extra credential. This
avoids consuming kernel command line space (which is limited) and
matches the approach used by mkosi.
Each mount is added as an fstab entry in the format:
{tag} {destination} virtiofs {ro|rw},x-initrd.mount
If the user already specified a fstab.extra credential via
--set-credential= or --load-credential=, the virtiofs mount entries
are appended to it rather than conflicting.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Add DISK_TYPE_SCSI_CD to support attaching disk images as CD-ROM
drives, needed for testing El Torito ISO images built by
systemd-repart.
When --image-disk-type=scsi-cd is specified, the image is attached
with media=cdrom and readonly=on on the drive, using scsi-cd as the
device driver on the SCSI bus. This also works for --extra-drive=
with the scsi-cd: prefix.
The QEMU configuration matches the standard OVMF CD-ROM boot setup:
-drive if=none,media=cdrom,format=raw,readonly=on
-device virtio-scsi-pci
-device scsi-cd
When direct kernel booting with scsi-cd, if the kernel command line
contains "rw", append "ro" to override it since CD-ROMs are
read-only.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
QEMU's stdio chardev sets O_NONBLOCK on both stdin and stdout (see
chardev/char-stdio.c [1] and chardev/char-fd.c [2]). Since forked
processes share file descriptions, and on a terminal all three stdio
fds typically reference the same file description, this affects our
own stdio too.
Avoid this by using a PTY with chardev serial instead of chardev
stdio for native console mode, matching the approach already used
for interactive and read-only modes. The PTY forwarder shovels bytes
transparently between our stdio and QEMU's PTY using the new
PTY_FORWARD_DUMB_TERMINAL and PTY_FORWARD_TRANSPARENT flags, which
disable terminal decoration (background tinting, window title, OSC
context) and escape sequence handling (Ctrl-] exit, hotkeys)
respectively.
The chardev is configured with mux=on so the QEMU monitor remains
accessible via Ctrl-a c.
Also dedup CONSOLE_NATIVE, CONSOLE_READ_ONLY, and CONSOLE_INTERACTIVE
handling by using fallthrough, with the only differences being the
ptyfwd flags, mux setting, and monitor section.
[1] https://gitlab.com/qemu-project/qemu/-/blob/master/chardev/char-stdio.c
[2] https://gitlab.com/qemu-project/qemu/-/blob/master/chardev/char-fd.c
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Pass -no-user-config while we're at it to avoid loading qemu config
from /etc which is more likely to cause hard to debug issues rather
than do something useful.
Reduce virtiofsd log noise by setting --log-level=error, and drop
the unnecessary mknod capability with --modcaps=-mknod, matching
mkosi's virtiofsd invocation.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Align find_ovmf_config() with mkosi's find_ovmf_firmware() by adding
checks that were previously missing:
- Filter on interface-types, only selecting UEFI firmware definitions.
Previously non-UEFI (e.g. BIOS-only) firmware could be selected.
- Check machine type compatibility using substring matching against the
target machine patterns in firmware descriptions (e.g. "q35" matches
"pc-q35-*"), following the same approach as mkosi.
- Make nvram-template optional in the firmware JSON mapping. Firmware
definitions without an nvram-template are now parsed successfully
(with vars remaining NULL) rather than failing entirely.
Also rework the firmware target parsing to store both architecture and
machine arrays per target (instead of just a flat architecture list),
and extract the machine matching into firmware_data_matches_machine().
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
It's useful to be able to check what firmware description vmspawn
will select. In particular, this will allow me to figure out the
nvram template file that will be picked up so I can pick it up in
mkosi and operate on it to pass a modified version of it to vmspawn
with --efi-nvram-template=.
Add --efi-nvram-template=PATH to specify a custom firmware variables
file to copy and use as the initial EFI NVRAM state instead of the
default template from the firmware definition.
Add --firmware-features=FEATURE[,FEATURE...] to require or exclude
specific firmware features during automatic firmware discovery.
Features prefixed with "!" are excluded. If a feature appears in both
the included and excluded lists, inclusion takes priority. Firmware
with the "enrolled-keys" feature is excluded by default.
Refactor --secure-boot= to operate on the firmware features sets
instead of maintaining a separate tristate. --secure-boot=yes adds
"secure-boot" to the include set, --secure-boot=no adds it to the
exclude set, and --secure-boot=auto removes it from both.
Generalize find_ovmf_config() to accept include/exclude feature sets
instead of a secure boot tristate, removing the special-cased
enrolled-keys and secure-boot filtering logic.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
vmspawn previously hardcoded virtio-blk for all drives. This adds
--image-disk-type= to select the root disk type (virtio-blk,
virtio-scsi, or nvme) and allows per-drive overrides via a
colon-separated prefix on --extra-drive=. The format and disk type
prefixes can appear in any order since their value sets don't overlap.
For virtio-scsi, a single shared controller is created with drives
attached as scsi-hd devices. For nvme, each drive gets its own
controller. Both have serial number length limits (30 and 20 characters
respectively), so long filenames are replaced with a truncated SHA-256
hex digest.
Extend --image-disk-type= and the --extra-drive= disk type prefix to
support nvme in addition to virtio-blk and virtio-scsi:
systemd-vmspawn --image-disk-type=nvme --image=image.raw
systemd-vmspawn --image=image.raw --extra-drive=nvme:data.raw
The NVMe serial number is limited to 20 characters by the NVMe spec.
If the image filename exceeds this, it is hashed with SHA-256 and
truncated to 20 hex characters via the disk_serial() helper introduced
in the previous commit.
Signed-off-by: Christian Brauner <brauner@kernel.org>
Add --image-disk-type= to select the disk type for the root disk, and
allow specifying the disk type as a colon-separated prefix on
--extra-drive=:
systemd-vmspawn --image-disk-type=virtio-scsi --image=image.raw
systemd-vmspawn --image=image.raw --extra-drive=virtio-scsi:data.raw
For --extra-drive=, the format and disk type prefixes can appear in any
order since the value sets don't overlap:
--extra-drive=raw:virtio-scsi:/path
--extra-drive=virtio-scsi:raw:/path
Extra drives inherit --image-disk-type= by default unless overridden
with an explicit prefix.
vmspawn originally used virtio-scsi for all drives but switched to
virtio-blk in 1f24a954e4 for simplicity and direct kernel boot
compatibility. This makes virtio-scsi available again as an explicit
option for cases where a SCSI storage topology is desired.
For virtio-scsi, a shared virtio-scsi-pci controller is created and
drives are attached as scsi-hd devices. The SCSI serial number is
limited to 30 characters, so filenames exceeding this are hashed with
SHA-256.
Signed-off-by: Christian Brauner <brauner@kernel.org>
It's unclear why I added this in fd05c6c759,
but it breaks bind mounting regular directories via --bind,
so drop it again since it's not actually required to make virtiofsd
work with the foreign UID range.
Typically, the top-level JSON object has to be an object, in any json
document we parse, hence let's add a simple way to enforce that.
Make use of this in various places.
(Note, various other JSON parsers insist on this logic right from the
beginning, but I actually thinking making this insisting optional like
this patch does it is the cleaner approach)
Closes#40877. As requested, --extra-drive=path[:format] is changed
to --extra-drive=[format:]path, so that the parsing is less ambiguous.
(In the original request, it was requested that the empty format can be
used also, but that was dropped in the second version of the patch.)
Zbigniew Jędrzejewski-Szmek
Daan De Meyer
Christian Brauner
Lennart Poettering
Mike Yuan
Daan De Meyer