This is the first varlink method added to sysupdate. The D-Bus interface
(via sysupdated) will remain for now; the varlink interface will exist
in parallel.
This method can be called via:
```
varlinkctl call ./path/to/systemd-sysupdate \
io.systemd.SysUpdate.CheckNew \
'{"target":{"class":"host"}}'
SYSTEMD_SYSUPDATE_NO_VERIFY=1 \
varlinkctl call ./path/to/systemd-sysupdate \
io.systemd.SysUpdate.CheckNew \
'{"target":{"class":"component","name":"some-component"}}'
```
This includes some changes to run the integration tests again using the
varlink interface rather than running `systemd-sysupdate` directly, to
test the new interface.
This adds the scaffolding for being able to call sysupdate via varlink,
but it doesn’t yet define or implement any methods. Those will come in
following commits.
The existing `systemd-sysupdate.service` and `systemd-sysupdate.timer`
(which periodically ran `systemd-sysupdate update`) have been renamed
to `systemd-sysupdate-update.{service,timer}` to make way for a new
`systemd-sysupdate@.service` and `systemd-sysupdate.socket` file to
handle varlink activation.
Compatibility symlinks have been added for them.
`process_image()` has historically used `umount_and_freep` to clean up
the mounted directory locally, but callers to it have used
`umount_and_rmdir_and_freep`.
No directory is created after any of the error return paths in
`process_image()`, so it should probably be using
`umount_and_rmdir_and_freep` too.
This is another step towards varlinkifying the program, as it means the
various verb implementations are no longer relying on global state from
the command line.
As part of this, move init of the `Context` struct into a new
`context_from_cmdline()` function.
Additionally pass some context into config parsing `userdata` arguments,
as various config parsers were using `arg_root` via a sneaky `extern`.
This introduces no functional changes.
There’s no need for it to be heap allocated — there’s only ever one
instance of it, and it’s allocated for the lifetime of a `verb_*()`
function.
Simplify things a bit by making it stack allocated. This will also help
with upcoming commits where we introduce derived context structs to help
with varlinkifying sysupdate. By allowing `Context` to be stack
allocated we can include it in the derived context structs.
As part of this, rename `context_make_{offline,online}()` to
`context_load_{offline,online}()` for clarity (since they no longer init
the struct).
This introduces no functional changes.
`process_image()` is always called immediately before (almost) every
`context_make_online()` or `context_make_offline()`, and the structures
it allocates have the same lifetime as `Context`, so we might as well
factor them all together to reduce duplication.
This will also simplify the following commit, which changes heap
allocation of `Context`s, and simplify upcoming changes to factor out
`arg_*` handling.
The call in `verb_pending_or_reboot()` is safe because it already
validates that `arg_image` is `NULL`, hence `process_image()` will bail
out early.
This introduces no functional changes.
This makes it like all the other verbs and therefore easier to refactor.
At the same time, remove the separate `component` argument and instead
use the `component` set on the `Context`. This guards against bugs, as
various parts of the `Context` state depend on the component (for
example, `installdb_fd`) and overriding the component without also
overriding its dependent variables will lead to bugs.
Define a new io.systemd.SysUpdate.Notify Varlink interface with a single
OnCompletedUpdate() method, and after sysupdate successfully installs an
update, invoke that method on every socket linked into
/run/systemd/sysupdate/notify/ via varlink_execute_directory(). This
gives other components a hook to react to applied updates (e.g. recompute
a TPM policy, link a freshly downloaded kernel, refresh extensions).
The notification carries the component name, the installed version and the
list of updated resources (transfer id + on-disk path). Subscribers are
free to ignore the parameters and just treat the call as a trigger.
Setting SYSTEMD_SYSUPDATE_FORCE_NOTIFY=1 forces the notification to be sent
even when no update was applied (in which case no resource list is included),
so follow-up work can be triggered unconditionally.
Fixes: #35988
This adds an operation equivalent to "systemd-sysupdate cleanup" after
an update completed (regardless if that update was entirely successful
or not). This ensures that any orphaned files are automatically cleaned
up, if they are not referenced by any transfer file's patterns anymore.
Follow-up for: d82e256bb9
Let's make sure we keep track of any file we drop into the system via a
database in /var/. This database is implemented based on symlinks, i.e.
reuses the fs as a simple database. Given the database most likely will
have <= 10 entries only (as we store *patterns* of installed file paths in
them, not the file paths themselves), this should be very efficient.
For implementation details see comments at top of
src/sysupdate/sysupdate-cleanup.c.
Just some refactoring.
The code is slightly updated, for example it now uses string_is_safe().
But mostly this just splits out code from one large file to a smaller
one.
Move the --reboot/--component= rejection into parse_argv() alongside the
other cross-option checks, and tighten TEST-72 to assert the specific
guard message rather than merely a non-zero exit.
The `pending` and `reboot` verbs, as well as the `--reboot` switch, compare
the newest installed version against the booted OS version (IMAGE_VERSION= from
os-release). When a component is selected via --component=, this compares the
component's version against the unrelated host OS version, which by design live
in separate version spaces. The result is arbitrary reboot decisions: depending
on the relative version strings sysupdate would either always or never reboot.
Refuse the combination with a clear error instead of silently performing a
bogus comparison. Correctly tracking a per-component booted version is left as a
future feature.
Fixes: https://github.com/systemd/systemd/issues/42330
`sysupdate --json=short components` lists the components known to
sysupdate; these are the components which something like `updatectl`
will try to update.
The `default` component represents the host, and is meant to be listed
if transfer definitions exist in (for example) `/etc/sysupdate.d`
corresponding to the host OS. This then corresponds to `TARGET_HOST` in
`updatectl` and causes it to try updating that target.
The logic for working out whether the `default` component was present
essentially boiled down to “does `{/run,/etc,/usr/lib}/sysupdate.d`
exist”, and it didn’t check whether a `.transfer` or `.conf` file
actually existed in the config directory.
This is quite the corner case, but becomes more evident on systems where
sysupdate is being used to update a portable service but not the main
OS. At that point, if `/etc/sysupdate.d` exists empty (for some reason),
`updatectl` falls over because it starts trying to update the host OS
without any configuration to do so.
So, modify `sysupdate` to more fully load the available configuration
when listing components, and query it a bit more deeply to check whether
a default component exists.
If `sysupdate` is called with various command line arguments to affect
how its configuration is loaded, do *not* say that a default component
exists, as these arguments essentially anull the possibility of a
default being used in that process.
Add an integration test based on the reproducer provided by the issue
reporter. This test has been tested to fail if the changes to
`sysupdate.c` aren’t applied — if so, the second call to `sysupdate
components` would return
`{"default":true,"components":["some-component"]}`.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41501
Optionally prevent `context_read_definitions()` erroring out if zero
transfer definitions were found.
This commit makes no functional changes (the flag is always passed to
calls to `context_make_offline()` for the moment), but the new flag will be
used in the following commit.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41501
This commit makes no functional changes, but the new flags will be
used and extended in the following commit. We need a flags variable to
avoid having two bool arguments, which would be confusing.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41501
I am really not a fan of full code lines passed to macros as parameters.
Let's get rid of the 3rd parameter of FOREACH_OPTION() hence:
1. Let's return errors just as a regular value (though a negative one),
that can be handled via a OPTION_ERROR case statement for the switch.
This normalizes handling of the error, just like any other event
returned by the option parser.
2. In order to avoid exploding the amount of boilerplate in each use
(that just propagates the error on OPTION_ERROR), let's then
introduce an explicit FOREACH_OPTION_OR_RETURN(), that returns from
the calling function on its own (and makes that clear in the name).
Together this cleans up, normalizes the logic and shortens the code.
It was requested to make the 'c', 'opt', and 'arg' params the same, i.e.
defined through the FOREACH_OPTION macro. But we can't do that easily,
because 'c' was defined in the for loop definition, and we can only
define variables of the same type in that way. Also, in some cases we
need only 'c', in other cases with need 'c' and 'arg, in some cases 'c'
and 'opt', and in other cases all three. We'd need to either
conditionalize or mark those variables with _unused_ to deal with
compiler warnings. But a different approach works quite nicely: add
state.opt and state.arg to show the current option and it's argument.
(The short names are picked on purpose to reduce verbosity since those
are used a lot.)
If an update is partially downloaded and the user tries to update again,
`updatectl` can’t currently do anything (it doesn’t yet support resuming
downloads). At the moment, though, it’ll return success as if the system
was up to date, even though it isn’t up to date.
Instead, print a more helpful error message telling the user to try
vacuuming the partial version and trying again.
I decided not to make it automatically vacuum the partial version, as
that seems like a way to get into a nasty retry loop if, for example,
the checksum provided by the server doesn’t match that of the downloaded
file (which is one way to trigger this code path).
Add an integration test which simulates this failure by corrupting the
`SHA256SUMS` file, trying to download an update, and then working
through the recovery steps.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41502
Previously we allowed a pending version to be the candidate — but if
there are no better choices, then we might as well allow a partial
version to be candidate as well.
The alternative is having no update candidate when a new version is
partially installed (i.e. downloaded but not moved into place). This
would mean that an update which is interrupted then needs to be re-run
with an explicit version number to progress, rather than being able to
be re-run without a version number (as it was in the first place).
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41502
While a resource Instance can either be partial or pending, but not
both; an UpdateSet (which potentially comprises several Instances) can
be both partial *and* pending if it contains Instances in both those
states.
Amend a few bits of internal code to allow that in situations which were
previously overlooked.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
If the resource code is recursing, it’s possible for one iteration to
set a partial flag, and then a recursive iteration to set a pending flag
(or vice-versa). It doesn’t make sense to have both set at the same time
for a specific instance, so make sure to clear the other flag when
setting one of them.
Add some assertions to make this invariant clearer and easier to debug
if it fails.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
`READY=1` is already correctly emitted when acquiring an update, but was
forgotten to be emitted when subsequently installing that update.
That meant that the state tracking code in `sysupdated` and hence
`updatectl` could not properly report the progress of the install
operation, and hence printed “Already up-to-date” after a successful
update installation, rather than “Done”.
Add a test to catch this in future.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41502
As mentioned in the grandfather commit, I want to use the _many
suffix for freeing of the contents of an array, so the functions
to free the array to get the suffix _array.
We often have a pattern where the same verb function is used for
multiple actions. This leads to an antipattern where we figure out what
action needs to be taken from argv[0] multiple times: often once in
parse_argv() to figure out what options are allowed, then once again
implicitly in dispatch_verb(), and then again in the action verb itself.
Let's allow passing a parameter into the verb to simplify this.
This matches a pattern we have in conf-parser.h, where we have both
void *userdata (more global) and void *data (per-config item). Here,
I opted for uintptr_t userdata. It seems that most of the time we'll
want to just pass an enum value. This works OK with no casts. I also
tried a void* and union. In both cases, much more boilerplate is needed:
either a cast or a macro to help avoid compiler warnings. uintptr_t
seems generic enough to cover foreseeable usecases with no fuss.
This is a noop refactoring. See next commit for an example.
We don't want to vacuum anything if we're just going to fail just
afterwards because a GPT partition label exceeds the maximum size
so let's compute the temporary paths for all transfers before we do
any vacuuming or acquiring.
These expose the two parts of ‘update’, so that update sets can be
acquired (downloaded) and installed (applied) in separate actions at
different times. For example, this could allow a load of update sets to
be acquired when online, and later applied when offline.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/34814
An ‘acquire’ (download) part, and an ‘install’ (apply) part.
Following commits will expose these as separate verbs and D-Bus methods,
but this commit is the one which rearranges the internals.
If doing an ‘install’, a mirror version of the ‘acquire’ has to happen
first to make sure the transfer’s internal state is correct.
‘Acquire’ can require an internet connection, but ‘install’ will always
work with `--offline` specified.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/34814
This commit adds the flags and some basic formatting/printing of them.
Following commits will integrate them into the update/acquire/install
logic.
`UPDATE_PARTIAL` is set if any of the instances in the `UpdateSet` are
partial, i.e. have been partially downloaded.
`UPDATE_PENDING` is set if any of the instances in the `UpdateSet` are
pending, i.e. have been acquired (downloaded) but not yet installed.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/34814
Now that sd-event doesn't insist on blocking
SIGCHLD anymore when watching for process exit,
let's stop blocking SIGCHLD tree-wide where not
required anymore.
This reverts commit 490aa05ca1.
As commented https://github.com/systemd/systemd/pull/38569#discussion_r2284978273,
the commit makes autofs check bypassed. Before the commit, when
CHASE_NO_AUTOFS is set, we did not shortcut chasing paths, and refused
any autofs mount points in the path. However, with the commit, the flag
was swapped but even when CHASE_AUTOFS is unset, the autofs check may be
skipped.
To fix the issue, rather than swapping the flag, we should introduce
another flag, say CHASE_TRIGGER_AUTOFS. This revert the commit, and in a
later commit, the new flag will be introduced.
Since https://github.com/systemd/systemd/commit/c5de7b14ae2e08d267d8d75bc88934ac6aa7dcd6
file searching implies a new mount api syscall by default,
to trigger automounts.
This is problematic in NSS plugins, as they are dlopen'ed inside
processes by glibc, for two reasons.
First of all, potentially searching on a networked filesystem
automount could lead to nasty surprises, such as the process
responsible for setting up the network filesystem trying to
search on that same filesystem.
More importantly, the new mount api syscall was never part of
the filesystem seccomp filter that we provide by default, and
given mounting/remounting/bind mounting is one of the possible
ways to bypass sandboxing it is very likely not allowed when
custom filters are used in sandboxed processes, if they don't
need to do these operations otherwise.
The filesystem seccomp mask we provide has been updated, however
this only takes effect on the next restart of a service. When
systemd is upgraded via a package upgrade, the new nss plugin is
installed and will be immediately dlopen'ed by glibc when needed,
without waiting for the process to restart, which means the existing
seccomp filter applies, causing the filter to trigger.
Given it's not really possible for any arbitrary program to
predict which NSS modules glibc will load, given programs do not
configure that and instead nsswitch is set up by the sysadmin,
it's impossible to handle at each process level. It's also not
possible to know when it will be triggered, given the plugin
is not linked in each binary tools like need-restart cannot
even pre-emptively restart services that may be affected.
This means in practice, upgrading from systemd << v258 to >= v258
requires a reboot to avoid either subtle or catastrophic system
failures.
By avoiding to trigger automounts in nss-systemd we can avoid
both issues.
userdb drop-ins are searched for in:
/etc/userdb/
/run/userdb/
/run/host/userdb/
/usr/local/lib/userdb/
/usr/lib/userdb/
none of which are supported as automounts anyway.
Note that this happens only when the userdbd service is not running,
as otherwise nss-systemd will go through the varlink IPC, rather than
doing the searches in-process.
So invert CHASE_NO_AUTOFS to CHASE_AUTOFS and set it in the places where
we do want to trigger automounts, like looking for the ESP.
Follow-up for https://github.com/systemd/systemd/commit/c5de7b14ae2e08d267d8d75bc88934ac6aa7dcd6
Fixes https://github.com/systemd/systemd/issues/38565
Philip Withnall
Lennart Poettering
Armaan Sandhu
Zbigniew Jędrzejewski-Szmek
Frantisek Sumsal
Daan De Meyer
Lennart Poettering
Yu Watanabe
Daan De Meyer
Luca Boccassi