Since being online is a privileged operation, it’s better to fail-safe
and make the `Context` offline by default unless explicitly initialised
(e.g. with `context_from_cmdline()` or an explicit initialisation of
`.offline = false`).
There are a few instances where `.offline = true` is set explicitly.
I’ve left those in place, even though they’re now redundant, as
documentation.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
(cherry picked from commit 97183383af03279c43d9c270af5832d36d527d01)
Currently, `CONTEXT_NULL` initialises `.offline` to false, so helper
contexts are being accidentally run in online mode. Fix that.
The following commit will rework this to be safe by default, but let’s
fix the individual instances first.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
(cherry picked from commit 4212fba4a4258a19057642d13a0ae52181675e23)
A verity signature partition carries a JSON object that must be followed by
NUL bytes only, up to the end of the partition (dissect refuses anything
else). The split-out artifact repart generates for it however is trimmed of
that padding since b54ef83414, so that it is a plain JSON text file (which
jq nowadays insists on).
sysupdate installs such a file into a partition by simply writing it at the
partition's offset, leaving whatever follows in place. Hence, once a shorter
signature (e.g. after rotating to a different key type) is installed into a
partition that previously held a longer one, the tail of the old signature
survives right after the new JSON object, and the partition becomes
unparsable.
Add a new PartitionZeroFill= boolean setting to the [Target] section, which
makes the importer continue writing zeroes up to the end of the partition
once the payload has been written, by passing --zero-fill= to systemd-import
and systemd-pull. It defaults to off, except for verity signature partitions
(which are tiny, hence this is cheap) where it defaults to on.
Fixes: #43567
(cherry picked from commit d3b6d765a5b320d6d864591c6a7b87e1047d0c78)
Similar as the previous commit: the field was renamed by accident, and
we should revert to the old name to not break compat needlessly, and
because the old name actually made sense, given the underlying .transfer
file knob is AppStream=
The "isEnabled" name was changed from "enabled" post v261 by accident
(because a6f7ded859 replaced json
formatting via table formatter to a native json generator which was
introduced for varlink, which changed the names). THis was later made
official in #43603, but it should rather have changed things back to the
original name, in particular as the underling .feature file field
actually is called "Enabled=" and not "IsEnabled=".
We really should take care to generate the same names for Varlink JSON
data as for the data returned by the systemd-sysupdate cli.
a6f7ded859 ensured that, so that's great.
But at the same time I think the varlink names should be adjusted to the
old names, since varlink support for this was just added in this cycle.
Given that v262 is not tagged yet, let's now revert to the old name.
For re-use in systemd-gpt-auto-generator. This also makes it generic
across sysext/confext - no planned use for this, but to avoid confusion
later when someone tries to use it on a confext.
Component definitions may contain multiple Documentation= URLs, but the
components command previously displayed only the first URL in text output.
Use TABLE_STRV_WRAPPED to display all component documentation URLs in the
table, making the full documentation list visible to users.
Add integration test coverage for multiple Documentation= assignments.
The feature JSON payload switched to `id`, `isEnabled`, and `appstream`,
and made `documentation` an array of URLs.
`updatectl` still parsed legacy table-derived names and a single URL.
This broke feature JSON parsing and made output inconsistent.
Features without documentation could also cause a null dereference.
Align `updatectl` with the current schema and preserve all
documentation URLs across sysupdate, D-Bus, and Varlink payloads.
Propagate condition evaluation errors through `condition_test_list()`
and `feature_is_suggested()`, with consistent JSON error handling and
tests.
The `suggested` field is now optional and always a boolean when
present; it is omitted rather than set to an error string when
SuggestOn*= conditions cannot be evaluated.
Add `suggested` metadata and document the field transition.
Follow-up for 3e34ccf6df.
Follow-up for 75a247d4de.
Follow-up for 40e727c93b.
Fixes#43603
Like the command-level CommandDescription.argspec, a verb's argspec is now
a nulstr, so that a verb may declare multiple alternative usage lines. The
verbs help table renders one synopsis line per nulstr entry in the
synopsis cell.
All verb argument specifications must gain an explicit NUL suffix. This
is a lot of churn, but I think this needs to be done. For now, there'll
be only one user of the new functionality (i.e. a verb with more than
one argspec), but I think we'll eventually want to use this in other
places too.
This converts systemd-homed, -hostnamed, -importd, -localed, -logind,
-machined, -networkd, -portebled, -resolved, -sysupdated, -timedated,
and -timesyncd.
The service name and description move from service_parse_argv() arguments
into a COMMAND() description that each daemon now defines. The
service_parse_argv() macro captures the calling binary's verb section,
where that COMMAND is found, while the options are still defined and
parsed in libshared, in the new "service" option namespace that each
COMMAND references. (The named namespace makes sure that options defined
by other translation units in libshared can never leak into the
services' parsers.)
The supported features are declared via option group filtering: each
COMMAND lists "Options" plus "Bus introspection" and/or "Runtime scope".
Unsupported options are now rejected by the parser as unknown options
(and are not shown in --help), replacing the runtime "This service does
not support …" errors.
The help output changes slightly: the "Bus introspection" and "Runtime
scope" option groups now get their own headings, and some descriptions
were reworded to the imperative form used elsewhere.
Almost all programs are simple and have only a single command. When
printing options, let's just use the first COMMAND block found by default.
This means we can drop the explicit program-name argument and reduce
copy&paste boilerplate.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added. The odd "Verbs" option group marker is dropped,
so that all options are listed in one section after the verbs, with --help
and --version first. The synopsis now uses the generic "[OPTION…] COMMAND …"
form: a bare VERSION is not accepted as the first argument, and the man
page does not list it either. --help now goes through the pager, like
other commands with --no-pager.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added. The "Options" group marker after --help/--version
is dropped, so that all options are listed in one section after the verbs.
The synopsis now uses the generic "[OPTION…] COMMAND …" form: a bare
VERSION is not accepted as the first argument, and the man page does not
list it either. --help now goes through the pager, like other commands
with --no-pager.
CurrentSymlink= accepts a symlink path, but the parser used NAME_MAX
for specifier expansion. Absolute and target-relative paths can be
longer than one filename component while still staying below PATH_MAX.
Reproducer: configure CurrentSymlink= to a path longer than NAME_MAX
and run systemd-sysupdate list with debug logging.
Before: sysupdate logged that CurrentSymlink= specifier expansion
failed and ignored the setting.
After: the transfer parses cleanly and list emits no CurrentSymlink=
specifier expansion warning.
Follow-up for: 43cc7a3ef4
138829b784 moved Documentation=, AppStream=, and related URL parsing
into sysupdate-config.c, but kept NAME_MAX as the specifier expansion
limit. These settings are URLs, not filenames.
Reproducer: configure a feature Documentation= URL longer than
NAME_MAX and run systemd-sysupdate features FEATURE.
Before: the URL was ignored after ENAMETOOLONG during specifier
expansion.
After: features FEATURE prints the long Documentation URL.
Follow-up for: 138829b784
systemd drop-ins use an empty assignment to reset list and scalar
settings. MinVersion= and ProtectVersion= treated an empty assignment
as an invalid version and kept the earlier value active.
Reproducer: define MinVersion=v2 and ProtectVersion=v1, then add a
drop-in with empty MinVersion= and ProtectVersion= assignments.
Before: sysupdate logged invalid-version warnings for the empty values.
After: list logs no invalid-version warnings and shows both v1 and v2.
Follow-up for: 43cc7a3ef4
sysupdate.d(5) documents PartitionFlags= as a hexadecimal integer, and
sysupdate patterns format partition flags with PRIx64. The parser used
safe_atou64(), which rejected values such as abcdef.
Reproducer: set PartitionFlags=abcdef in a partition target transfer
and run systemd-sysupdate list with debug logging.
Before: sysupdate logged:
Failed to parse partition flags, ignoring: abcdef
After: the transfer parses cleanly and list emits no partition-flags
parse warning.
Follow-up for: 43cc7a3ef4
48fff6a27a switched filesystem transfers to predictable
.sysupdate.partial.* names, but transfer_remove_temporary()
still only scanned the legacy .# prefix.
Remove the current partial prefix there too. Pending instances
are deliberately left for transfer_vacuum(), so ProtectVersion=
can still spare protected pending versions.
Follow-up for: 48fff6a27a
sysupdate.d(5) documents linux-generic as the default
MatchPartitionType= for partition targets. After 10eaca4159,
omitting the option left partition_type_set false and update failed
before acquiring the image.
Reproducer: create a partition target transfer without
MatchPartitionType= and run systemd-sysupdate update.
Before: update failed with:
Partition type must be set for partition targets.
After: update succeeds on a linux-generic slot and installs the
version, leaving the GPT label as part-v1.
Follow-up for: 10eaca4159
The disabling of features or components with the flag
--component-suggested/--feature-suggested didn't disable the suggested
ones but instead disabled all other ones. This is rather unintuitive due
to how the flags are named and also not really needed because the
intended reconciliation outcome can instead be done by first disabling
everything and then enabling the suggested ones again which is easier to
reason about. For components the tricky part is that they default to
enabled and thus it's better to have the disable/enable commands with
--component-suggested operate only on suggested ones instead of touching
others like "legacy" components that don't explicity say whether they
are enabled and suggested or not.
Make running disablement of components/features with
--component-suggested/--feature-suggested undo a previous enablement
with the same flags. Document how one can align the system to only use
suggested components/features and not anything else by doing it in two
steps, first disabling everything and then enabling suggested ones. This
also makes it clearer now that all components that are not explicitly
enabled nor suggested will be disabled then.
--component-all is documented to include the default component-less
installation. Do not drop it merely because the context operates on a
root/image, or because all its transfers are currently disabled by
features.
This lets --component-all --feature-all enable-feature write the
default component feature drop-ins instead of succeeding with no
components selected.
TEST-72-SYSUPDATE covers both all transfers disabled by features and
the same default component feature operation under --root=.
Repro: create a default feata.feature plus a transfer gated by feata,
then run:
build/systemd-sysupdate --root="$root" --component-all --feature-all enable-feature
Before: no drop-in was written.
Follow-up for: 4481661a75
When recording installdb entries under --root=, keep the leading slash
after stripping the root. Compare current transfer target paths in the
same root-relative form during cleanup.
This prevents cleanup from treating still-owned resources below --root=
as orphaned.
TEST-72-SYSUPDATE covers --root= cleanup keeping a still-owned file and
its matching installdb entry.
Repro: create a rooted transfer for /target/foo-@v.bin, add a
matching installdb entry for /target/./foo-@v.bin, then run:
build/systemd-sysupdate --root="$root" --verify=no cleanup
Before: foo-1.bin and the installdb entry were removed.
Follow-up for: d82e256bb9
Definitions enumerated under --root= are already rooted. Passing those
paths to the config parsers with the same root prefixes the root again,
so feature and transfer files are parsed from the wrong path.
Repro: create root/etc/sysupdate.d/rootfeat.feature and
01-root.transfer, then run:
build/systemd-sysupdate --root="$root" --verify=no --offline features rootfeat
Before: parsing failed at line 1 with a bogus Source Type= error.
Fixes#42783.
Follow-up for: e1384cfb09
Return failure for --json check-new when no candidate is available.
Let sysupdated accept that exact JSON no-update result from workers.
Reproducer: systemd-sysupdate --verify=no --json=short check-new
Before: the command printed {"available":null} and exited successfully.
Follow-up: 42c0b689a8
Since 4c0d8d9673, most dlopen notes are
set at the beginning of the executables. Let's manage all dlopen notes
there, rather than setting them where dlopen is called.
Note that the only exceptions are the LIBBPF_NOTE for networkd and
nsresource. Since dlopen_bpf() is wrapped in an `#if` guard, the notes
are instead set within the corresponding functionality.
As a result, the DLOPEN_FOO() wrapper macros are no longer needed and
can be dropped completely.
Following on from adding the basic varlink scaffolding to sysupdate,
let’s varlinkify a couple of the D-Bus methods. Because varlink doesn’t
have a concept of object paths, the D-Bus path structure which allows a
target to be selected has been squashed down to a target argument for
each relevant method.
Varlinkify the way to list targets, and also the way to list features
because that was simple to do at the same time.
More methods need varlinkifying in the future, but let’s do it in small
and manageable chunks.
Let's take inspiration from string_is_safe() and take a flags field that
allows fine tuning the validation.
Then port over all current users of either function to the new logic.
Note that this *does* change behaviour in various cases:
1. Generally: we'll now always accept the full UAPI.10 alphabet,
including the "~" and "^" characters. As far as I can see there's no
downside to this liberalization as none of the current consumers of
the two functions uses these characters for anything else.
2. systemd-analyze compare-version will now accept version strings with
"_" and "+" without complaining. I see no downside here, it just
normalizes these debugging tools, to make them accept what most our
other tools accept.
3. "bootctl link" will not accept empty version strings anymore
Which is a bugfix I guess.
4. vpick will now refuse "_" and "+" in version strings. It kinda
already did, because when parsing versions from filenames it uses "_"
and "+" as name, architecture and attempt counter separators. We now
systematically refuse it everywhere else in vpick too. This is hence
a clean-up.
Fixes: #28906
Replaces: #42815#41937
Change the varlink API for Feature structs to allow multiple
documentation URLs for them, to match what systemd already does for
units etc.
This is a deviation from what the sysupdated D-Bus API allows and, for
the moment, from what’s supported internally by sysupdate. Internally it
continues to support 0-1 URLs for now.
But by defining the API as a strv, multiple URLs can be supported in
future without API breaks.
Historically, the ListFeatures API (in both D-Bus and now varlink) was
run without an `--offline` flag. This appears like it’s an oversight, but
actually `verb_features()` always unconditionally loaded in offline
mode.
In any case, there doesn’t appear to be a reason for the context to be
online (i.e. for it to check sources for available updates). Features are
defined in local config files and are loaded by `read_features()`, which
is called in both offline and online mode.
When the varlink ListFeatures API was added, it was put into online mode
in order to match the lack of `--offline` argument in the existing D-Bus
API implementation. Change both of them to be explicitly in offline mode.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Since we now enumerate all targets on a varlink call (to validate the
requested target), this message gets printed multiple times in the log.
It’s not really necessary, so downgrade it to a debug message.
list VERSION printed the version status header twice in plain output.
Keep the complete later header and add a regression check.
Follow-up for: 42c0b689a8.
This adds similar concepts as we already have for enable-component:
let's add a way to enable all features or the suggeste dones, and
possibly on all components.
Or in other words, with this:
systemd-sysupdate enable-component -S
systemd-sysupdate enable-feature -A -s
We'll automatically enable all suggested components, and all features of
them.
Let's now introduce "systemd-sysupdate enable-component
--component-suggested" and systemd-sysupdate enable-component
--component-all" for enabling all or all suggested components at once.
Similar, if used for disable-component will disable all components or
those not suggested.
Philip Withnall
Lennart Poettering
Kai Wohlfahrt
dongshengyuan
Zbigniew Jędrzejewski-Szmek
felix h
Luca Boccassi
Yu Watanabe
Kai Lüke