We nowadays support unprivileged invocation of systemd-nspawn +
systemd-vmspawn, but there was no support for discovering suitable disk
images (i.e. no per-user counterpart of /var/lib/machines). Add this
now, and hook it up everywhere.
Instead of hardcoding machined's, importd's, portabled's, sysupdated's
image discovery to RUNTIME_SCOPE_SYSTEM I introduced a field that make
the scope variable, even if this field is always initialized to
RUNTIME_SCOPE_SYSTEM for now. I think these four services should
eventually be updated to support a per-user concept too, this is
preparation for that, even though it doesn't outright add support for
this.
This is for the largest part not user visible, except for in nspawn,
vmspawn and the dissect tool. For the latter I added a pair of
--user/--system switches to select the discovery scope.
We'd log that we're skipping the target, but it would never actually get
removed from the manager's list. Thus, we'd advertise targets that don't
actually exist to clients.
In the original version of the sysupdated PR, this was handled by
removing the target from the manager's list in target_free, and using a
_cleanup_ attribute to free the target when skipping. However, this
changed at some point during review. So, this commit takes the
alternative approach
This commit introduces a build-time option to enable/disable sysupdated
separately from sysupdate. 'auto' translated to enabled by default in
developer builds.
We often used a pattern like if (!FLAGS_SET(flags, SD_JSON_FORMAT_OFF)),
which is rather verbose and also contains a double negative, which we try
to avoid. Add a little helper to avoid an explicit bit check.
This change clarifies an aditional thing: in some cases we treated
SD_JSON_FORMAT_OFF as a flag (flags & SD_JSON_FORMAT_OFF), while in other cases
we treated it as an independent enum value (flags == SD_JSON_FORMAT_OFF).
In the first form, flags like SD_JSON_FORMAT_SSE do _not_ turn the json
output on, while in the second form they do. Let's use the first form
everywhere.
No functional change intended.
Initially I wasn't sure if this helper should be made public or just internal,
but it seems such a common pattern that if we expose the flags, we might just
as well expose it too, to make life easier for any consumers.
Optional features allow distros to define sets of transfers that can
be enabled or disabled by the system administrator. This is useful for
situations where a distro may want to ship some resources version-locked
to the core OS, but many people have no need for the resource, such as:
development tools/compilers, drivers for specialized hardware, language
packs, etc
We also rename sysupdate.d/*.conf -> sysupdate.d/*.transfer, because
now there are more than one type of definition in sysupdate.d/. For
backwards compat, we still load *.conf files as long as no *.transfer
files are found and the *.conf files don't try to declare themselves
as part of any features
Fixes https://github.com/systemd/systemd/issues/33343
Fixes https://github.com/systemd/systemd/issues/33344
First, this fixes a case where an error is logged twice at the LOG_ERR
level. Second, this goes through and raises the log level where
appropriate so that sysupdated is easier to debug. Finally, it replaces
EINVAL with EPROTO where appropriate, since EINVAL implies that the
caller passed some incorrect arguments (which is incorrect; sysupdate
passed some invalid JSON back to us)
If there are no updates available, `updatectl check` would previously
just print an empty table. We should instead just tell the user that
there are no updates available.
In mkosi, I want to add a sysupdate verb to wrap systemd-sysupdate.
The definitions will be picked up from mkosi.sysupdate/ and passed
to systemd-sysupdate. I want users to be able to write transfer
definitions that are independent of the output directory used by
mkosi. To make this possible, it should be possible to specify the
directory that transfer sources should be looked up in on the sysupdate
command line. Let's allow this via a new --transfer-source= option.
Additionally, transfer sources that want to take advantage of this
feature should specify PathRelativeTo=directory to indicate the configured
Path= is interpreted relative to the tranfer source directory specified
on the CLI.
This allows for the following transfer definition to be put in
mkosi.sysupdate:
"""
[Transfer]
ProtectVersion=%A
[Source]
Type=regular-file
Path=/
PathRelativeTo=directory
MatchPattern=ParticleOS_@v.usr-%a.@u.raw
[Target]
Type=partition
Path=auto
MatchPattern=ParticleOS_@v
MatchPartitionType=usr
PartitionFlags=0
ReadOnly=1
"""
If someone runs `updatectl update`, sysupdate will be running multiple
update jobs at the same time, which can make reasoning about the output
in the journal quite difficult. Especially if things go wrong: the error
messages didn't mention which job failed. Nor was there any link between
job ID and the PID of the worker process logging to the journal. This
is all fixed here!
Cuts out some `strdup`s, and also avoids a rather weird case of donating
memory to a function. Basically just duplicates the solution I just
implemented for sysupdate's callout handler.
Previously, if the callout binary (i.e. sd-pull, sd-import) failed
gracefully, we'd return its exit status from the event loop and thus
from run_callout(). Of course, exit status is a positive number in the
event of failure. Which means that we completely ignore the callout
binary failing, and instead continue using whatever it managed to
download before failing.
This is bad for obvious reasons, not the least of which is installing
a half-downloaded OS. This also means that we would completely ignore
failed signature checks 😬️
The progress_bar functions do their own buffering: they reconfigure
stderr, then print, then flush and disable buffering on their own. In
situations where multiple progress bars are being drawn at a time (for
example, in updatectl), it's even more efficient to hoist the buffering
and flushing to the call site, and avoid drawing each progress bar
individually.
To that end, new _unbuffered variants of the progress_bar functions. And
we use them in updatectl.
This applies a couple of aesthetic changes to the way updatectl renders
progress information
1. We invert from "ICON TARGET MESSAGE" to "TARGET: ICON MESSAGE" to
better fit in with the systemd progress bars, which look like
"TARGET [==========---------] XX%". The original version of the
sysupdated PR implemented its own progress bars that were oriented
differently: "[==========---------] TARGET XX%". When we swapped
the progress bar we didn't swap the status messages
2. When a target finishes updating, instead of leaving a 100% progress
bar on screen for potentially extended periods of time (which implies
to the user that the update isn't actually done...), we show a status
message saying the target is done updating.
3. Fixed a minor bug where an extra newline would be printed after the
total progress bar. At the top of the rendering function, we scroll
the terminal's scroll-back just enough to fit a line for each target,
and one for the total. This means that we should not print an
additional line after the total, or else it'll scroll the terminal's
buffer by an additional character. This bug was introduced at some
point during review
4. Clears the Total progress bar before quitting. By the time we're
quitting, that progress bar will be showing no useful status for the
user. Also, the fix in point 3 will cause the shell's prompt to
appear on the same line as the Total progress bar, partially
overwriting it and leaving the shell in a glitchy state.
This fixes a bug introduced during review of sysupdated. Originally,
we just returned EALREADY verbatim to signify that the target is
already up-to-date. Then we switched this to a proper error
(org.freedesktop.sysupdate1.NoCandidate) during review. But that now
maps to EIO, not EALREADY. Thus, whenever there's nothing to update,
updatectl would report I/O errors to the user, even though nothing
actually went wrong.
A previous commit made sysupdate recognize installed versions where some
transfers are missing. This commit teaches sysupdate how to correctly
repair these incomplete versions.
Previously, if you had a incomplete installation of the OS booted, and
ran sysupdate in an attempt to repair it, sysupdate would make things
worse by creating copies of the currently-booted partitions in the
inactive slots. Then at boot you have two identical partitions, with
identical labels an UUIDs, and end up with a mess.
With this commit, sysupdate is able to recognize situations where it can
simply download the missing transfers and leave the rest of the system
undistrubed.
Partial fix for https://github.com/systemd/systemd/issues/33339
When enumerating what versions exist for a given target, sysupdate would
completely throw out any version that's incomplete (where some of the
transfers in the target have that version installed or available, and
other transfers do not).
If we're trying to find what versions we can offer for download, this is
great behavior. If the server side is advertising a partial update to
download, we shouldn't present it to the user.
On the other hand, if we're enumerating what versions we have currently
installed, this is a bad behavior. It makes sysupdate fragile. For
example, if a sysext introduces a new .conf file into
/usr/lib/sysupdate.d, suddenly the currently-installed OS stops being a
version that we've enumerated. Since it's not enumerated, it's not
protected, and so sysupdate will wipe the booted OS.
So if we're looking for installed versions, we now loosen the
restrictions and enumerate incomplete installations.
Partial fix for https://github.com/systemd/systemd/issues/33339
The current implementation will never find a match, because in the event
of a match instance_cmp falls through to comparing paths and the key
we're matching against will always have a path of NULL.
So let's just use a separate compare function, just to make sure future
updates to instance_cmp don't break resource_find_instance again.
Lennart Poettering
Matthias Lisin
Abderrahim Kitouni
Yu Watanabe
Adrian Vovk
Zbigniew Jędrzejewski-Szmek
Antonio Alvarez Feijoo
Daan De Meyer
Mike Yuan