Follow-up to #42422:
- Rename process_image() to context_process_image(), since it now
operates on a Context object.
- Use IN_SET() in image_type_can_sysupdate() instead of a switch.
- Name the return parameters of context_list_components() ret_xyz, per
our coding style.
- Drop a redundant "else" after a return in vl_method_check_new().
This is the first varlink method added to sysupdate. The D-Bus interface
(via sysupdated) will remain for now; the varlink interface will exist
in parallel.
This method can be called via:
```
varlinkctl call ./path/to/systemd-sysupdate \
io.systemd.SysUpdate.CheckNew \
'{"target":{"class":"host"}}'
SYSTEMD_SYSUPDATE_NO_VERIFY=1 \
varlinkctl call ./path/to/systemd-sysupdate \
io.systemd.SysUpdate.CheckNew \
'{"target":{"class":"component","name":"some-component"}}'
```
This includes some changes to run the integration tests again using the
varlink interface rather than running `systemd-sysupdate` directly, to
test the new interface.
This adds the scaffolding for being able to call sysupdate via varlink,
but it doesn’t yet define or implement any methods. Those will come in
following commits.
The existing `systemd-sysupdate.service` and `systemd-sysupdate.timer`
(which periodically ran `systemd-sysupdate update`) have been renamed
to `systemd-sysupdate-update.{service,timer}` to make way for a new
`systemd-sysupdate@.service` and `systemd-sysupdate.socket` file to
handle varlink activation.
Compatibility symlinks have been added for them.
This will be used in upcoming commits to varlinkify `systemd-sysupdate`;
it will need a way to identify targets over varlink, and the existing
way with a `Target` over D-Bus seems to work quite well.
`process_image()` has historically used `umount_and_freep` to clean up
the mounted directory locally, but callers to it have used
`umount_and_rmdir_and_freep`.
No directory is created after any of the error return paths in
`process_image()`, so it should probably be using
`umount_and_rmdir_and_freep` too.
This is another step towards varlinkifying the program, as it means the
various verb implementations are no longer relying on global state from
the command line.
As part of this, move init of the `Context` struct into a new
`context_from_cmdline()` function.
Additionally pass some context into config parsing `userdata` arguments,
as various config parsers were using `arg_root` via a sneaky `extern`.
This introduces no functional changes.
There’s no need for it to be heap allocated — there’s only ever one
instance of it, and it’s allocated for the lifetime of a `verb_*()`
function.
Simplify things a bit by making it stack allocated. This will also help
with upcoming commits where we introduce derived context structs to help
with varlinkifying sysupdate. By allowing `Context` to be stack
allocated we can include it in the derived context structs.
As part of this, rename `context_make_{offline,online}()` to
`context_load_{offline,online}()` for clarity (since they no longer init
the struct).
This introduces no functional changes.
`process_image()` is always called immediately before (almost) every
`context_make_online()` or `context_make_offline()`, and the structures
it allocates have the same lifetime as `Context`, so we might as well
factor them all together to reduce duplication.
This will also simplify the following commit, which changes heap
allocation of `Context`s, and simplify upcoming changes to factor out
`arg_*` handling.
The call in `verb_pending_or_reboot()` is safe because it already
validates that `arg_image` is `NULL`, hence `process_image()` will bail
out early.
This introduces no functional changes.
This makes it like all the other verbs and therefore easier to refactor.
At the same time, remove the separate `component` argument and instead
use the `component` set on the `Context`. This guards against bugs, as
various parts of the `Context` state depend on the component (for
example, `installdb_fd`) and overriding the component without also
overriding its dependent variables will lead to bugs.
If the FD has already been opened, return 1 as if opening was
successful, rather than returning 0 as if it gave `ENOENT`.
This fixes doing multiple installdb operations on a single `Context`.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Both vacuum loops silently return 0 on failure instead of propagating
the error code, hiding disk cleanup failures from the caller.
Signed-off-by: dongshengyuan <dongshengyuan@uniontech.com>
Define a new io.systemd.SysUpdate.Notify Varlink interface with a single
OnCompletedUpdate() method, and after sysupdate successfully installs an
update, invoke that method on every socket linked into
/run/systemd/sysupdate/notify/ via varlink_execute_directory(). This
gives other components a hook to react to applied updates (e.g. recompute
a TPM policy, link a freshly downloaded kernel, refresh extensions).
The notification carries the component name, the installed version and the
list of updated resources (transfer id + on-disk path). Subscribers are
free to ignore the parameters and just treat the call as a trigger.
Setting SYSTEMD_SYSUPDATE_FORCE_NOTIFY=1 forces the notification to be sent
even when no update was applied (in which case no resource list is included),
so follow-up work can be triggered unconditionally.
Fixes: #35988
This adds an operation equivalent to "systemd-sysupdate cleanup" after
an update completed (regardless if that update was entirely successful
or not). This ensures that any orphaned files are automatically cleaned
up, if they are not referenced by any transfer file's patterns anymore.
Follow-up for: d82e256bb9
Let's make sure we keep track of any file we drop into the system via a
database in /var/. This database is implemented based on symlinks, i.e.
reuses the fs as a simple database. Given the database most likely will
have <= 10 entries only (as we store *patterns* of installed file paths in
them, not the file paths themselves), this should be very efficient.
For implementation details see comments at top of
src/sysupdate/sysupdate-cleanup.c.
Just some refactoring.
The code is slightly updated, for example it now uses string_is_safe().
But mostly this just splits out code from one large file to a smaller
one.
Move the --reboot/--component= rejection into parse_argv() alongside the
other cross-option checks, and tighten TEST-72 to assert the specific
guard message rather than merely a non-zero exit.
The `pending` and `reboot` verbs, as well as the `--reboot` switch, compare
the newest installed version against the booted OS version (IMAGE_VERSION= from
os-release). When a component is selected via --component=, this compares the
component's version against the unrelated host OS version, which by design live
in separate version spaces. The result is arbitrary reboot decisions: depending
on the relative version strings sysupdate would either always or never reboot.
Refuse the combination with a clear error instead of silently performing a
bogus comparison. Correctly tracking a per-component booted version is left as a
future feature.
Fixes: https://github.com/systemd/systemd/issues/42330
unhexmem_full() ignores whitespace, so a 64-byte manifest digest field
can decode to fewer than 32 bytes. Reject that while parsing instead.
[ 83.883087] TEST-72-SYSUPDATE.sh[5995]: systemd-sysupdate: ../src/src/sysupdate/sysupdate-resource.c:581: resource_load_from_web: Assertion `h.iov_len == sizeof(instance->metadata.sha256sum)' failed.
Follow-up for 43cc7a3ef4
Assisted-by: kres (claude-opus-4-7)
Co-developed-by: Claude Opus 4.8 <noreply@anthropic.com>
Currently almost all the dlopens happen in libbasic or libshared code,
so the ELF dlopen notes all end up in libsystemd-shared. Many
distributions split this library and various binaries in separate
packages, and the library ends up with soft-dependencies, even though
many binaries are either completely useless or do not work at all with
the dlopen dependency. This also makes it impossible to know which
executable uses which dlopen dependency without inspecting the source
code.
If someone only wants to add the soft dependencies from libshared they
can just avoid parsing the executable binaries. By design the code in
libbasic/libshared still does the stamping too, at lower priorities, so
that libsystemd-shared will always list all the optional dependencies,
and if one wants to build a minimal system by default, they can just
parse libsystemd-shared dlopen notes, and ignore the individual
executables. But for many distribution the current setup is insufficient
and requires adding a ton of manual library dependencies, as many
executables become effectively broken or useless without the dlopen
dependencies installed (eg: resolved fails to start without libssl,
repart can do basically nothing without blkid, etc).
Add a new set of DLOPEN_<LIB> macros that wrap the dlopen_lib and also
pull in the ELF note voodoo, so that the callers get their ELF binaries
stamped too. Convert a bunch of callers to use the macro, and use
`required` dependencies for the callers that do not work without the
dlopen library being available.
The one caveat is that, in order to avoid duplicating the exact same
note in a binary due to multiple call sites, some `asm` voodoo is done
instead of the previous bare-C section-creating macro. The drawback of
this approach is that if `--gc-sections` is used to link the binary (as
we do), then binutils >= 2.36 is required for the `SHF_GNU_RETAIN` flag.
This effectively cuts off CentOS 9, so what I did here is adding an
override in meson to detect missing support, and drop `SHF_GNU_RETAIN`.
The build works, but on CentOS 9 there's no dlopen ELF notes anymore.
Given it's just that version, and it goes EOL next year, that seems ok
to me. The alternative is to drop the usage of `--gc-sections` on CentOS
9, or to accept duplicated notes everywhere, and both seem worse.
End result:
```
$ readelf --notes build/systemd-executor
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0
Properties: x86 ISA needed: x86-64-baseline
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 8a0c3db54adb79ae54e1432255011aa4ab583742
Displaying notes found in: .note.ABI-tag
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 3.2.0
Displaying notes found in: .note.dlopen
Owner Data size Description
FDO 0x0000006b FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"pam","description":"Support for LinuxPAM","priority":"recommended","soname":["libpam.so.0"]}]
FDO 0x0000007c FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"seccomp","description":"Support for Seccomp Sandboxes","priority":"recommended","soname":["libseccomp.so.2"]}]
FDO 0x00000090 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"bpf","description":"Support firewalling and sandboxing with BPF","priority":"recommended","soname":["libbpf.so.1","libbpf.so.0"]}]
FDO 0x000000a0 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"cryptsetup","description":"Support for disk encryption, integrity, and authentication","priority":"recommended","soname":["libcryptsetup.so.12"]}]
FDO 0x00000078 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"mount","description":"Support for mount enumeration","priority":"recommended","soname":["libmount.so.1"]}]
$ readelf --notes build/systemd
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0
Properties: x86 ISA needed: x86-64-baseline
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: dcd4568842e32da3e71be27db3def51c6b459994
Displaying notes found in: .note.ABI-tag
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 3.2.0
Displaying notes found in: .note.dlopen
Owner Data size Description
FDO 0x00000075 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"mount","description":"Support for mount enumeration","priority":"required","soname":["libmount.so.1"]}]
FDO 0x00000072 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"selinux","description":"Support for SELinux","priority":"recommended","soname":["libselinux.so.1"]}]
```
Switch the executable-owned call sites from the plain dlopen_<lib>()
helpers to the new DLOPEN_<LIB>() macros, so that each binary that loads
an optional library at runtime now carries its own .note.dlopen note
instead of relying solely on libsystemd-shared. Each call site passes a
priority reflecting whether it hard-depends on the library (required) or
degrades gracefully without it (recommended).
Co-developed-by: Claude Opus 4.8 <noreply@anthropic.com>
`sysupdate --json=short components` lists the components known to
sysupdate; these are the components which something like `updatectl`
will try to update.
The `default` component represents the host, and is meant to be listed
if transfer definitions exist in (for example) `/etc/sysupdate.d`
corresponding to the host OS. This then corresponds to `TARGET_HOST` in
`updatectl` and causes it to try updating that target.
The logic for working out whether the `default` component was present
essentially boiled down to “does `{/run,/etc,/usr/lib}/sysupdate.d`
exist”, and it didn’t check whether a `.transfer` or `.conf` file
actually existed in the config directory.
This is quite the corner case, but becomes more evident on systems where
sysupdate is being used to update a portable service but not the main
OS. At that point, if `/etc/sysupdate.d` exists empty (for some reason),
`updatectl` falls over because it starts trying to update the host OS
without any configuration to do so.
So, modify `sysupdate` to more fully load the available configuration
when listing components, and query it a bit more deeply to check whether
a default component exists.
If `sysupdate` is called with various command line arguments to affect
how its configuration is loaded, do *not* say that a default component
exists, as these arguments essentially anull the possibility of a
default being used in that process.
Add an integration test based on the reproducer provided by the issue
reporter. This test has been tested to fail if the changes to
`sysupdate.c` aren’t applied — if so, the second call to `sysupdate
components` would return
`{"default":true,"components":["some-component"]}`.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41501
`sysupdate --json=short components` lists the components known to
sysupdate; these are the components which something like `updatectl`
will try to update.
The `default` component represents the host, and is meant to be listed
if transfer definitions exist in (for example) `/etc/sysupdate.d`
corresponding to the host OS. This then corresponds to `TARGET_HOST` in
`updatectl` and causes it to try updating that target.
The logic for working out whether the `default` component was present
essentially boiled down to “does `{/run,/etc,/usr/lib}/sysupdate.d`
exist”, and it didn’t check whether a `.transfer` or `.conf` file
actually existed in the config directory.
This is quite the corner case, but becomes more evident on systems where
sysupdate is being used to update a portable service but not the main
OS. At that point, if `/etc/sysupdate.d` exists empty (for some reason),
`updatectl` falls over because it starts trying to update the host OS
without any configuration to do so.
So, modify `sysupdate` to more fully load the available configuration
when listing components, and query it a bit more deeply to check whether
a default component exists.
If `sysupdate` is called with various command line arguments to affect
how its configuration is loaded, do *not* say that a default component
exists, as these arguments essentially anull the possibility of a
default being used in that process.
Add an integration test based on the reproducer provided by the issue
reporter. This test has been tested to fail if the changes to
`sysupdate.c` aren’t applied — if so, the second call to `sysupdate
components` would return
`{"default":true,"components":["some-component"]}`.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41501
Optionally prevent `context_read_definitions()` erroring out if zero
transfer definitions were found.
This commit makes no functional changes (the flag is always passed to
calls to `context_make_offline()` for the moment), but the new flag will be
used in the following commit.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41501
This commit makes no functional changes, but the new flags will be
used and extended in the following commit. We need a flags variable to
avoid having two bool arguments, which would be confusing.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41501
This allows us to have a separate, more permissive, policy for
cancelling ongoing sysupdate jobs. The new default policy for
cancellation actions is to allow them for the active user, without admin
authentication, because typically the user can just pull the plug on the
computer to cancel a job anyway.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/38568
This allows us to have a separate, more permissive, policy for
cancelling ongoing sysupdate jobs. The new default policy for
cancellation actions is to allow them for the active user, without admin
authentication, because typically the user can just pull the plug on the
computer to cancel a job anyway.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/38568
While commit 83c1e8ff5f added support for interactive polkit
authentication in `updatectl`, some users might want to disable that for
some use cases; so add the standard `--no-ask-password` argument.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/37412
- **bpf-util: rename from bpf-dlopen, unify version-specific symbol
handling**
- **cryptsetup: dlopen libcryptsetup in tokens**
- **tree-wide: dlopen libpam in pam plugins**
- **test-bus-marshal: dlopen() glib and libdbus instead of linking
directly**
- **lock-util: Simplify timeout for lock_generic_with_timeout()**
- **color-util: simplify hsv_to_rgb, fix rgb_to_hsv negative-hue wrap**
- **tree-wide: Use our own macros instead of fabs()/fmax()/fmin()**
- **locale-util: dlopen() libintl instead of linking against it**
- **home: Use log2u64() over log2()**
- **meson: drop libdl, threads, and librt dependencies**
- **libc: Use dlsym() from a constructor instead of weak symbols**
- **libc: Make sure C23 versions of strtol(), sscanf() are not used**
Our baseline glibc is 2.34, which merged libdl, libpthread (the
dependency('threads') target), and librt into libc. Empty .so/.a stubs
remain for backward compatibility with old binaries, but new builds
resolve dl_*, pthread_*, mq_*, timer_*, etc. directly from libc.
On musl the same libraries are likewise empty stubs.
Drop the libdl, threads, and librt entries from every meson.build, and
remove the now-stale 'Libs.private: -lrt -pthread' from libudev.pc.in
since both flags resolve to empty link-time stubs on glibc 2.34+ and
musl.
Verified with readelf -d that libsystemd.so, libudev.so, and systemd no
longer carry DT_NEEDED entries for libdl/libpthread/librt.
This was missing in the CurrentSymlink= creation path, and leads to
partially-broken update installs.
Signed-off-by: Aleksa Sarai <aleksa@amutable.com>
An error snuck into the pattern parsing of the `MatchPattern` key in the
sysupdate transfer files. If there's two files "part1-v2.raw", and
"part1-v2.raw.tar" in the source folder, and MatchPattern="part1-@v.raw",
sysupdate will incorrectly choose "part1-v2.raw.tar" instead of
"part1-v2.raw".
While the pattern matching works perfectly fine, after the full pattern
is successfully matched to the string, we don't ensure that the string
actually ends when the pattern just did.
This means we can end up choosing a wrong file for the update, if the
filename/path happens to start with the same MatchPattern.
Fix it by ensuring the string ends after our match pattern ended.
I am really not a fan of full code lines passed to macros as parameters.
Let's get rid of the 3rd parameter of FOREACH_OPTION() hence:
1. Let's return errors just as a regular value (though a negative one),
that can be handled via a OPTION_ERROR case statement for the switch.
This normalizes handling of the error, just like any other event
returned by the option parser.
2. In order to avoid exploding the amount of boilerplate in each use
(that just propagates the error on OPTION_ERROR), let's then
introduce an explicit FOREACH_OPTION_OR_RETURN(), that returns from
the calling function on its own (and makes that clear in the name).
Together this cleans up, normalizes the logic and shortens the code.
It was requested to make the 'c', 'opt', and 'arg' params the same, i.e.
defined through the FOREACH_OPTION macro. But we can't do that easily,
because 'c' was defined in the for loop definition, and we can only
define variables of the same type in that way. Also, in some cases we
need only 'c', in other cases with need 'c' and 'arg, in some cases 'c'
and 'opt', and in other cases all three. We'd need to either
conditionalize or mark those variables with _unused_ to deal with
compiler warnings. But a different approach works quite nicely: add
state.opt and state.arg to show the current option and it's argument.
(The short names are picked on purpose to reduce verbosity since those
are used a lot.)
If an update is partially downloaded and the user tries to update again,
`updatectl` can’t currently do anything (it doesn’t yet support resuming
downloads). At the moment, though, it’ll return success as if the system
was up to date, even though it isn’t up to date.
Instead, print a more helpful error message telling the user to try
vacuuming the partial version and trying again.
I decided not to make it automatically vacuum the partial version, as
that seems like a way to get into a nasty retry loop if, for example,
the checksum provided by the server doesn’t match that of the downloaded
file (which is one way to trigger this code path).
Add an integration test which simulates this failure by corrupting the
`SHA256SUMS` file, trying to download an update, and then working
through the recovery steps.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41502
Previously we prevented partial and pending versions from being
vacuumed. But until we support resuming downloads, there’s nothing else
which can be done with a partial version except to vacuum it and try
again.
Accordingly, allow partial versions (but not pending versions) to be
vacuumed.
This behaviour can be changed again once resuming downloads is supported
— at that point I expect we’ll want to try resuming the partial download
rather than throwing it all away and trying again.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41502
Previously we allowed a pending version to be the candidate — but if
there are no better choices, then we might as well allow a partial
version to be candidate as well.
The alternative is having no update candidate when a new version is
partially installed (i.e. downloaded but not moved into place). This
would mean that an update which is interrupted then needs to be re-run
with an explicit version number to progress, rather than being able to
be re-run without a version number (as it was in the first place).
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41502
While a resource Instance can either be partial or pending, but not
both; an UpdateSet (which potentially comprises several Instances) can
be both partial *and* pending if it contains Instances in both those
states.
Amend a few bits of internal code to allow that in situations which were
previously overlooked.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Lennart Poettering
Philip Withnall
dongshengyuan
Armaan Sandhu
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
Daan De Meyer
Aleksa Sarai
Jonas Dreßler