Verify that a DNS-SD service registered via the D-Bus RegisterService()
call is bound to the lifetime of the client's bus connection: since
busctl disconnects right after the call returns, the service must be
unregistered again shortly after. Wait for resolved's vanish handler
log message as proof that the client tracking triggered the
unregistration, check that a subsequent UnregisterService() fails
specifically because the service is not known anymore, and verify that
the identifier can be registered again — previously it stayed taken by
the vanished client's service forever, failing with
DNSSD_SERVICE_EXISTS.
A dummy link with mDNS enabled makes sure the per-scope goodbye/zone
teardown path actually runs, and since resolved would be restarted
automatically if it crashed in it, letting the D-Bus level checks pass
vacuously, verify the InvocationID stayed the same throughout.
The custom help() is replaced by a COMMAND() description, matching the
other report-sign services. The abstract is now wrapped at 80 columns
in the help output.
Use the new copy timestamp clamping mechanism from the previous commit
to fix unreproducible wall-clock files in the generated images.
That covers both modes: online mode writes through the kernel, and
offline mode stages the tree in a temporary directory before handing it
to mkfs or mcopy. `mcopy -m` then carries the clamped mtimes into the
FAT.
This complements commit 34c3d57474 (directory times) and commit
f191ca982c (FAT insertion order).
Cover all combinations (online/offline, with and without epoch) in the
new `testcase_vfat_reproducibility()`. Update `testcase_make_symlinks()`
to cover the symlink creation/re-fix their parent directory code paths.
Sadly, images are still not reproducible byte for byte. Document
systemd-repart's $SOURCE_DATE_EPOCH handling and remaining
reproducibility issues in the manpage.
systemd-udevd is the same binary as udevadm, so the introspection data
describes both commands, from either name. The COMMAND intentionally
owns no verbs — all of udevadm's verbs are defined in udevadm.c right
after its own COMMAND, so that they are attributed to it.
The -V/--version output (GIT_VERSION) is preserved.
The custom help() is replaced by the common verb help, which derives the
synopsis, abstract, and option list from the verb declaration in
udevadm.c and the "udevadm-control" option namespace. The synopsis
makes use of the new nulstr-argspec functionality:
> udevadm [OPTION…] lock [OPTION…] COMMAND
> udevadm [OPTION…] lock [OPTION…] --print
The custom help() is replaced by the common verb help, which derives the
synopsis, abstract, and option list from the verb declaration in udevadm.c
and the "udevadm-control" option namespace. The new VERB_DEPRECATED flag
is used.
The verbs declare their subcommand option namespaces, so the
introspection data describes the options of every subcommand, nested in
the verb objects.
The bare version number output (a dracut compatibility requirement) is
preserved for --version/-V and the "version" verb.
The "lock" verb's argument specification is simplified to "COMMAND" --
other verbs do not mention their options in the argspec either, and the
subcommand help states where the options may go.
The final enablement pieces needed for coco tests in systemd CI. Runners
will be incus containers, these two commits enable running coco tests
with mkosi boot (where image is a directory), then start a confidential
guest with vmspawn as part of the test.
Add a TEST_MKOSI_ARGS environment variable whose contents are appended
to the mkosi invocation launching the test image, without affecting the
image build. This lets the environment adapt how tests are run, e.g.
pass --format=directory --output-dir=... to boot the test container
from a directory format output in environments that cannot dissect disk
images, such as unprivileged containers, where loop devices and udev
are not available.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
In a mount namespace owned by a non-initial user namespace, the kernel
only allows mounting a fresh proc or sysfs if the namespace already
contains a fully visible mount of the same filesystem type
(mount_too_revealing()). systemd-nspawn's pivoted root contains no such
mount, so booting the test container inside an unprivileged container
(e.g. a containerized CI runner) fails with EPERM when mounting /proc.
When running in a non-initial user namespace, satisfy the check by
binding a read-only instance of each into the container under /work.
Skipping the binds on regular privileged runs keeps the container
isolated from the host's proc and sysfs. Binding non-recursively keeps
foreign submounts (e.g. lxcfs overmounts) out, which would spoil full
visibility where they end up locked, and which read-only would not
cover.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
UpdateHomeEx needs to set up the home temporarily so that it can
write the refreshed identity records. Inspect and password changes use
the same setup path for metadata-oriented work.
For LUKS homes with luksDiscard=no, home_setup_luks() fully
allocates the backing image while setting it up. If diskSize=max was
previously written to a sparse image, this can fail with ENOSPC before
the user can disable auto-resize mode again.
Add a setup flag that lets metadata operations skip the full-image
fallocate step. Real create, activate, and resize paths keep the
existing allocation behavior.
Before, reproduced on Ubuntu 26.04 with /home on tmpfs:
homectl create reprohomed \
--storage=luks \
--fs-type=btrfs \
--disk-size=300M \
--auto-resize-mode=shrink-and-grow \
--luks-discard=no
homectl update reprohomed --disk-size=max
dd if=/dev/zero of=/var/tmp/homed-repro/filler bs=1M count=500
systemctl restart systemd-homed.service
homectl update reprohomed --auto-resize-mode=off
Operation on home reprohomed failed:
Not enough disk space for home reprohomed
Fixes#34527
Follow-up for: ab3b6fcb40
When a partition is deleted from a table containing multiple partitions
of the same type, filling the resulting partition-number gap can place
a new partition before an existing partition matched to an earlier
definition. This changes definition matching on the next invocation and
breaks idempotency.
Keep new partitions after existing partitions of the same type. If no
partition table slot is available after them, fall back to the first
unused slot.
Which existing partition is matched to which definition is still
determined by partition type and partition number order, as documented,
and is unchanged by this.
Fixes#43449
When an administrator installed a custom keyring under /etc,
signature verification selected it instead of the vendor keyring
under /usr. That could make vendor-signed system updates stop
verifying after adding local keys.
Collect all readable default keyrings, including the current and
legacy /etc paths, and pass temporary copies to GnuPG with
separate --keyring options. Keep SYSTEMD_OPENPGP_KEYRING as an
explicit override, add regression coverage for keyring
coexistence, override behavior, and current-path signing, and
silence the expected shellcheck false positive in the test
cleanup helper.
Closes: #40702
The transactions-with-cycles subtest starts 20 mutually cyclic
services, reads the manager's TransactionsWithOrderingCycle ids over
varlink, and then requires a journal entry matching "cycle starting
with" for every id. On a fedora rawhide CI run the entry for
transaction id 42 was missing for the entire boot and journalctl
printed "-- No entries --", failing the unit, while the manager had
correctly recorded all 20 ids. In the exported journal of that boot,
transaction 42 kept only its debug-level enqueue line
transaction-cycle12.service: Trying to enqueue job
transaction-cycle12.service/start/replace
while the other 19 transactions each also kept the three messages
that follow it ("Found ordering cycle", "Unable to break cycle
starting with", "unfixable cyclic ordering dependency"), and PID1's
next message, accepted about 2 ms after the enqueue line, shows
journald itself kept ingesting.
The messages were lost, not late. The test VM boots with
systemd.log_level=debug, and the 20 cyclic starts made PID1 emit
about 12,400 debug messages within 30 seconds, 4,100 of them in the
final second, during which journald stalled for 94 ms between two
consecutive entries. PID1 gives its journal socket a 10 ms send
timeout (src/basic/log.c), and a message whose send times out is
either dropped outright, because log_struct() ignores the sendmsg
result and falls back only on formatting errors, or, for the plain
log calls that do fall back to kmsg, reappears as _TRANSPORT=kernel
text stripped of its structured fields, where a TRANSACTION_ID=
match cannot find it. The asserted messages are log_struct() calls,
so they are in the dropped class. Either way the journalctl
assertion comes up empty. This is also why the "journalctl --sync"
added in ed0727fecd does not help here: it only flushes what journald
already received.
Run the whole section, including the daemon-reload, at log level
info, and drain what the earlier subtests queued at debug level with
one "journalctl --sync" before the first start. The daemon-reload is
itself a heavy debug emitter that returns before journald has
drained its burst, so opening the quiet window after it would leave
that backlog competing with the first cycle messages. The two
messages the subtest greps for are emitted at LOG_ERR ("Unable to
break cycle starting with") and LOG_WARNING ("Job ... deleted to
break ordering cycle starting with") in src/core/transaction.c, and
the ids are recorded into transactions_with_cycle on the same code
path directly before them, so the assertion keeps its full strength
while the debug flood that displaced it is gone.
A single at_exit handler installed at the start of the test restores
the boot-time log level when any subtest aborts, and cleans up the
transient unit the aborting subtest had in flight, guarded on the
unit file existing as in TEST-26-SYSTEMCTL.sh. It replaces the
per-subtest restart_randomized_delay_cleanup() handler, so the two
subtests that change the log level share one restore path.
systemd-shutdown parsed a bunch of options, but had no way to print
them. Add the standard --help and --version options and the CLI
introspection.
parse_argv() is moved before the PID 1 check, so that the new options
also work when the binary is invoked manually. As PID 1 we can never
return from main() (the kernel would panic), so in that case parse
errors still freeze(), and the help output is followed by a freeze()
too. Ctrl-Alt-Delete and console reopening are re-enabled before
freezing, so that the freeze can be escaped through the kernel and the
error is visible, as before.
The --help/--version/introspect-cli option names are already listed in
the kernel command line exclusion list in proc-cmdline.c.
--introspect-cli follows the same deferred pattern as --help: parse_argv()
only records the action, and it is executed in the same place where
ACTION_HELP and ACTION_VERSION are handled.
The help is looked up by explicit name: in the multicall configuration
the systemd and systemd-executor commands share one binary, so the
"first COMMAND" default would depend on link order.
The "This program takes no positional arguments." line is dropped from
the help output — the default "[OPTION…]" synopsis already says so.
(And also, this is not quite true. It does take positional args when
called as init.)
This converts systemd-homed, -hostnamed, -importd, -localed, -logind,
-machined, -networkd, -portebled, -resolved, -sysupdated, -timedated,
and -timesyncd.
The service name and description move from service_parse_argv() arguments
into a COMMAND() description that each daemon now defines. The
service_parse_argv() macro captures the calling binary's verb section,
where that COMMAND is found, while the options are still defined and
parsed in libshared, in the new "service" option namespace that each
COMMAND references. (The named namespace makes sure that options defined
by other translation units in libshared can never leak into the
services' parsers.)
The supported features are declared via option group filtering: each
COMMAND lists "Options" plus "Bus introspection" and/or "Runtime scope".
Unsupported options are now rejected by the parser as unknown options
(and are not shown in --help), replacing the runtime "This service does
not support …" errors.
The help output changes slightly: the "Bus introspection" and "Runtime
scope" option groups now get their own headings, and some descriptions
were reworded to the imperative form used elsewhere.
Standard parse_argv() func is added. --version is now accepted
(previously only the help forms were recognized, via argv_looks_like_help()).
The synopsis changes from "systemd-integritysetup COMMAND ..." to the
standard "systemd-integritysetup [OPTION…] COMMAND …", and the help output
gains an Options section. "help" as the first argument keeps working via
the hidden common help verb.
Previously, we'd just print help and exit with success if no arguments
were given. This seems unexpected and wrong. The program now prints the
usual error message:
Command verb required (one of attach, detach, help).
A newly created network namespace may disallow ICMP echo sockets for
unprivileged users even when their GIDs are mapped into the container.
When creating both user and network namespaces, initialize
net.ipv4.ping_group_range using the mapped GID range capped at INT32_MAX.
This follows the system-wide default from sysctl.d/50-default.conf while
respecting the user namespace mapping and the kernel's accepted limit.
Leave externally supplied network namespaces untouched and treat write
failures as non-fatal warnings.
Add regression coverage for the default 64K mapped range using the tools
already available in the minimal container image.
Fixes#41603
This time, various commands with non-trivial requirements are handled.
So the series is shorter, but has a pattern of feature additions and
then those new features being used. The way that --help output is
formatted is changed a bit in various places. Those changes are
described in individual commits.
22 'static int help()' functions left after this.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added.
Options are now in two groups: "Common options" and Mount options". In
--help, for systemd-mount, both groups are shown. For systemd-umount, only
the first. This changes the behaviour a bit: previously, both groups were
shown together as one list. I think the new behaviour is actually a bit
better, because it makes it easier for the user to figure out that the
commands share common options and one of the commands has its private
second group. But if people think that the previous behaviour should
be restored, we could add a display options to squish the groups back
together.
--help now goes through the pager, like other commands with --no-pager.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added. --help now goes through the pager, like other
commands with --no-pager.
Also, previously, -h/--help/--version were listed in the "Commands"
section. This was cute, but recreating this in the new scheme would
be complicated. Also, it seems unnecessary. --help/--version and some
other commands terminate the program in all binaries, so they could
be considered "commands" in all cases. But we printed them together
with verbs only in a few cases. So I think it's fine to move them
back to the "Options" section. This makes the description of options
much more consistent between different programs.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added. The note about invocation as 'installkernel' is
kept as the footer. Verb and option column widths are kept separate, since
the verb table is very wide. --help now goes through the pager, like other
commands with --no-pager.
A separate COMMAND block is included for installkernel, but it is only
used for introspection data.
The custom halt_help() and shutdown_help() are replaced by COMMAND()
descriptions (one per command name, halt/poweroff/reboot sharing the
"halt" option namespace), and --introspect-cli is added to both parsers.
halt/poweroff/reboot now each reference their own man page name.
Create the file with open_tmpfile_linkable_at().
Write the recovery key and trailing newline to the temporary fd.
Set the final mode to 0600 before publishing it.
Link it into place with link_tmpfile_at(..., LINK_TMPFILE_SYNC).
As discussed in the PR, those fields can go down, so they cannot be advertised
as type COUNTER. But the type is shared for the whole family, so we need to
create a new family to keep the other ones as COUNTER and add the new ones
as GAUGE.
kernel/sched/cputime.c says above get_cpu_idle_time_us() and get_cpu_iowait_time_us():
> Note that this is partially broken due to the counter of iowait tasks that
> can be remotely updated without any synchronization. Therefore it is possible
> to observe backward values within two consecutive reads.
steal is apparently also unreliable because it is reported by the hyperviser.
So let's be cautious and report it as gauge too.
Extend the io.systemd.Basic metric families with host-level runtime
metrics: cumulative per-state CPU seconds, memory in use (MemTotal minus
MemAvailable), swap in use, the kernel's 10s pressure stall averages
(cpu/memory/io x some/full). Partitions and stacked devices are excluded
from the disk counters, since their traffic is already accounted for in
the parent device.
Also extend the TEST-74 report subtest with checks for the new metrics.
Network metrics are not added, because the service runs with
PrivateNetwork=yes, and it doesn't make sense to report stats for 'lo'.
After some back-and-forth, I settled on "CPUUsage". We use this scheme of
capitalization in other metric family names: "IPv4Public", "IPv6Public",
IPv4AddressState", "IPv6AddressState", "OSRelease" "SMBIOS" "BootID",
"CPUsOnline", "MachineID", "MachineInfo", "OSRelease",
"SMBIOS.SystemManufacturer", "TPM2.Manufacturer", "TPM2.VendorString",
"IOReadBytes", "IOReadOperations", "CGroup". The exception is
"io.systemd.cgroup.CpuUsage". I think we should treat that one as a
mistake and consider changing it to conform.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added. The generic help-verb check in the test is
filtered out, since systemctl's "help" verb shows unit manuals and fails
without arguments.
Arch rolling now ships Nmap 7.991, which contains Nmap commit
b408fc2, titled "Ignore stdin with recv-only"[1].
That change omits stdin from the Ncat descriptor list in recv-only
mode. For a Unix listener, the listening socket is then its sole
descriptor. Ncat enters its listen loop only when that list has more
than one descriptor or a connection already exists, so it exits
immediately after bind and leaves the socket pathname behind.
systemd-oomd sees that stale pathname, tries to invoke the hook, and
gets ECONNREFUSED. The notification output therefore remains empty
and the jq assertion fails. The problem is visible in trunk's CI
failures[2].
Use socat for these one-way Unix listeners. It remains in accept while
waiting for the hook and is already installed in integration images.
[1] https://github.com/nmap/nmap/commit/b408fc243ce26da7ca497595833c14ce36a5e1a4
[2] https://github.com/systemd/systemd/actions/runs/32622985037/job/97153925154
The DBus API already exposes the parsed machine-tags list via the Tags
property but varlink clients needed to parse the raw TAGS= line from
MachineInformationData (which also required implementing tag name
validation if the tags would later be used with SetTags, to avoid
spurious errors).
So, provide the same information that DBus gets (a list of pre-parsed
tags with bad tags gracefully removed) as a new MachineTags field
returned by Describe -- this does mean that the DBus API now exposes the
same data in two ways but making Describe() consistent is more important
and mirrors the OperatingSystem* class of properties.
image_rename() updated the Image object name and path before
computing the source paths for auxiliary files. As a result, the
auxiliary rename loop looked for files with the new image name,
while existing sidecars such as .roothash, .verity, and
.raw.tpmstate still used the old image name.
Delay updating the Image object until after renaming its settings
and auxiliary files. This lets image_auxiliary_path() resolve
existing sidecars from the old image name while
rename_auxiliary_file() assigns the new name.
Extend the import test raw image rename coverage to check
representative sidecars are renamed together with the image.
UnsetEnvironment= was applied to the environment block before the
command line was expanded. Since the expansion looks the variables up
in that same block, any variable listed in UnsetEnvironment= was
already gone when $VAR in ExecStart= and friends was expanded, and a
whole-word $VAR reference was dropped from the command line.
Let the expansion see the full environment block, and apply the
unsetting only to the environment block passed to the executed
process. This way UnsetEnvironment= can hide variables from the
executed process without breaking command lines that reference them.
Extend the exec-unsetenvironment test accordingly: the whole-word
$VAR6 argument must be expanded into a positional parameter for the
shell, while ${VAR6-X} must evaluate to X in the executed process.
Also document in both the UnsetEnvironment= description and the
"Environment Variables in Spawned Processes" section that the
unsetting happens after command line variable expansion.
Fixes: #43343
This adds an initial TPM2 implementation of the io.systemd.Report.Signer
interface, which returns a set of signed TPM attestations that also
includes the supplied report digest.
The returned report consists of multiple components:
- A signed PCR quote generated with TPM2_Quote.
- Signed NvPCR attestations generated with TPM2_NV_Certify.
- A signed audit session attestation generated with
TPM2_GetSessionAuditDigest.
Each signed report component includes the JSON encoded TPMS_ATTEST
attestation structure and corresponding signature, in both PEM encoded
form and as a JSON encoded TPMT_SIGNATURE structure.
NvPCR report components also include the readable name of the NvPCR, the
NV index public area as the JSON encoded TPMS_NV_PUBLIC structure, and some
additional authenticated data that is digested and supplied as the
qualifying data to the TPM2_NV_Certify command. This authenticated data
includes the NvPCR name and priority.
The report digest that is supplied via varlink is supplied as the
qualifying data to the audit session attestation.
The returned report also includes the event log which is retrieved from
pcrlock's varlink interface, and the public area of the signing key in
both PEM encoded form and as a JSON encoded TPMT_PUBLIC structure.
The audit session attestation is always the last component. The
preceding components are all executed with the audit session, and the
signer requires the audit session to remain exclusive whilst generating
the report. What this does is provide evidence to a verifier that the
overall report is a single, consistent snapshot of the system (ie, no
other TPM commands executed whilst generating the report) because the
audit session attestation contains a field that indicates that the
session was exclusive at the time it was attested.
The set of attested PCRs and NvPCRs are currently hardcoded. It includes
all firmware PCRs with the exception of PCR6, and all other PCRs with
the exception of the IMA one (which isn't useful on its own without the
IMA log). All defined NvPCRs are included.
A verifier can verify the signed attestation components individually by
verifying their signatures against the supplied attestation data, using
the supplied public key.
For the PCR report component, a verifier can replay the measurements in
the log in order to reconstruct the PCR values, and then use this and
the PCR selection from the attestation in order to calculate the PCR
digest, and then check that this matches the one in the attestation.
For the NvPCR report components, a verifier checks that the NV index
name in the attestation matches one that is anchored in PCR9 to be sure
that the attestation is for a valid NvPCR. A verifier can then replay
the measurements in the log to ensure that the resulting NvPCR value
matches the one supplied in the attestation. The authenticated data is a
JSON blob containing the NvPCR name and priority. A verifier can digest
this and ensure it matches the qualifying data in the attestation.
The audit session attestation includes the audit session digest and
whether the audit session was still exclusive when the attestation was
generated. A verifier should confirm exclusivity. The digest includes a
record of all TPM commands executed with the audit session, as each
command results in it being extended with the command parameter hash and
the result parameter hash. A verifier should replay the preceding report
components to compute an expected audit digest and make sure that this
digest matches the one in the attestation in order to be sure that the
overall report represents a single, consistent snapshot. The other
report components provide sufficient information for a verifier to
reconstruct the cpHash and rpHash.
When reconstructing the cpHash for the PCR and NvPCR components, the
signing scheme is the NULL scheme (they use the attestation key's
signing scheme). The required signing key public area is supplied in
the report. The supplied qualifying data is in the extra data field of
the attestation.
The PCR selection is also required to reconstruct the cpHash for the
PCR component. The PCR selection is available in the attestation.
The name of the NV index is also required to reconstruct the cpHash for
the NvPCR components. The name is available in the attestation.
The rpHash for each component can be reconstructed from the supplied
attestation blob and signature.
In the future, we might want to add more report component types. Eg, we
could add report components for ordinary NV indexes. Using the audit
session means that we could also add additional components that the TPM
can't attest directly, such as the values of interesting TPM properties
(as the TPM2_GetCapability calls will appear in the audit session digest).
The attestation signing key is stored as a JSON file in the report
signer's state directory. Different types of keys are supported:
- Ordinary keys that are protected by an existing persistent TPM key
such as the SRK or EK.
- Persistent TPM keys.
- Primary keys, which are recreated using a template on use.
When an ordinary or primary key is used, the report signer saves the
object context in its runtime directory. This context is used on future
invocations. This has the benefit that where a primary key is used, it
doesn't need to be recreated on each use. When an ordinary key is used,
we don't need to run TPM2_Load, which requires authorization with the
parent key, on each This authorization currently requires a policy
session for low-range EKs.
Regardless of the key type, it must be a restricted signing key. If no
key exists, an ordinary key is generated automatically.
The attestation key can also have an associated credential which is
attached to the signed report. The credential can be issued by a CA or
some other party during onboarding, and is presented to a verifier as
proof that the attestation key is authentic. It is considered an opaque
blob in order to not require it to be a specific format, such as a X.509
certificate. I've called this a "voucher" though because "cred" or
"credential" has other meanings in systemd.
As a follow-up to this, I plan to add a varlink interface for creating
attestation keys that can be used for report signing.
Stefan Agner
Yu Watanabe
Zbigniew Jędrzejewski-Szmek
Martin Pitt
felix h
Luca Boccassi
Paul Meyer
dongshengyuan
quanyeyang
zouge666
EJ Campbell
totoro
Nick Rosbrook
Jakub Stasiak
Aleksa Sarai
lucia-huang7
tunaichao
Chris Coulson