This is a lot of stuff, and sometimes quite wild, let's turn this into
its own header.
All stuff color-related that just generates sequences is now in
ansi-color.h (no .c file!), and everything more complex that
probes/ineracts with terminals remains in termina-util.[ch]
In previous commits, we've changed the JSON name mangling logic. This,
of course, will cause breaking changes to occur on anything that relied
on the JSON mangling logic.
This commit fixes those breaking changes by manually forcing the JSON
name back to what it was before.
It's time. sd-json was already done earlier in this cycle, let's now
make sd-varlink public too.
This is mostly just a search/replace job of epical proportions.
I left some functions internal (mostly IDL handling), and I turned some
static inline calls into regular calls.
Previously, importd was only accessible via D-Bus, which required it to
be a late boot service. Now that we have Varlink we can rearrange things
to become early-boot activated, just after the image directories are
mounted.
This will later allow us to have generator that auto-downloads images on
boot.
This is preparation for making our Varlink API a public API. Since our
Varlink API is built on top of our JSON API we need to make that public
first (it's a nice API, but JSON APIs there are already enough, this is
purely about the Varlink angle).
I made most of the json.h APIs public, and just placed them in
sd-json.h. Sometimes I wasn't so sure however, since the underlying data
structures would have to be made public too. If in doubt I didn#t risk
it, and moved the relevant API to src/libsystemd/sd-json/json-util.h
instead (without any sd_* symbol prefixes).
This is mostly a giant search/replace patch.
If we destroy both an event loop and a curl contect object at the same
time, then we get into this weird situation where curl wants us to
reconfigure a timout event source right before destruction, which
sd-event will refuse however, since it is already being shutdown.
Hence, catch that and simply don't bother adjusting the timeout, since
we cannot get back from there anyway.
gcrypt is used only for journal sealing operations in libsystemd, so it
can be made into a dlopen dependency that is used only on demand. This
allows to reduce the footprint of libsystemd in the most common cases.
Keep systemd-pull and systemd-resolved with normal linking, as they are
executables, and usually built with OpenSSL support anyway.
Let's make fd_verify_safe_flags() even more useful:
1. let's return the cleaned up flags (i.e. just the access mode) after
validation, hiding all the noise, such as O_NOFOLLOW, O_LARGEFILE and
similar.
2. let's add a "full" version of the call that allows passing additional
flags that are OK to be set.
It's a bit weird we allow importing/pulling/exporting images, but we
have no scheme for showing what#s already downloaded. Hence let's add
this, it's easy to add after all.
Let's downgrade log levels a bit on HTTP error codes. After all we
gracefully handle many of them, and we do generated an extra message for
the ones which are fatal anyway, hence there's no point in emphasizing
the HTTP erro message levels as we currently do.
Previously, when downloading an image, importd would first download them
into one image which it would then consider immutable (named after the
originating URL/etag), and then immediately make a copy of it (named
after the client chosen name).
This makes some sense in VM/container cases where the images are
typically mutable, and thus the original downloaded copy is of some
value.
For sysexts/confexts/portable this doesn't make much sense though, as
they are typically immutable. Hence make the concept optional.
This adds --keep-download=yes/no as a new option that controls the
above. Moreover it disables the behaviour for all image classes but
"machine". The behaviour remains enabled for "machine", for compat.
A while back we introduced image_name_is_valid() for validating image
file names. It's more liberal than hostname_is_valid() in many ways (and
allows version suffixes and such). Since importd deals in offline images
(as opposed to machined otherwise which deals in running machines),
let's hence use the right helper to validate the identifiers.
This adds "Ex" versions of all bus calls import implements, that make
two changes:
1. A "class" parameter is added that allows choosing between
machine/sysext/confext/portable images to download. Depending on the
chose class the target directory is selected differently (i.e. not
just /var/lib/machines/, but alternatively /var/lib/portables/,
/var/lib/extensions/, /var/lib/confexts/.
2. The boolean flags are replaced by a 64bit flags parameter.
The two enums are mostly the same, the former is just an extension of
the latter. Let's merge them, to simplify things. This is particularly
useful as we then can reuse this systematically as D-Bus method call
flags too, in a generic fashion that works for both imports and pulls
the same.
Pretty much just renaming of flags.
This is pretty much a 1:1 copy of the importd specific part of
machinectl.
We turn this into a separate tool, so that we can eventually make the
tool generic to also download other DDIs, not just machine images.
if we try to open file:// URLs that don't exist, we'll not get IO/timer
events about it, hence it is not sufficient to check for completion in
these events. Let's add a defer event, to deal with that.
Also, curl_multi_info_read() is a queue, make sure to handle all events
that might be queued.
This simplifies bus_verify_polkit_async() and related calls quite a bit:
1. This removes any support for authentication-by-Linux-capability. This
is ultimately a kdbus leftover: with classic AF_UNIX transports we
cannot authenticate by capabilities securely (because we cannot
acquire it from the peer without races), hence we never actually did.
Since the necessary kernel work didn't materialize in the last 10y,
and is unlikely to be added, let's just kill this context. We cannot
quite remove the caps stuff from sd-bus for API compat, but for our
polkit logic let's kill it.
2. The "good_uid" and "interactive" params are only necessary in very
few cases, hence let's move them to a new call
bus_verify_polkit_async_full() and make bus_verify_polkit_async() a
wrapper around it without those two parameters.
This also fixes a bunch of wrong uses of the "interactive" bool. The
bool makes no sense today as the ALLOW_INTERACTIVE_AUTHORIZATION field
in the D-Bus message header replaces it fully. We only need it to
implement method calls we introduced prior to that header field becoming
available in D-Bus. And it should only be used on such old method calls,
and otherwise always be set to false.
This does not change behaviour in any way. Just simplifies stuff.
Fixes: #21586
Since we parse it on the other side via parse_percent() which requires
that, otherwise we get an error:
[ 8.133131] testsuite-13.sh[649]: + machinectl import-raw /tmp/container.raw container-raw
[ 8.175035] machinectl[1143]: Enqueued transfer job 1. Press C-c to continue download in background.
[ 8.182130] machinectl[1143]: Importing '/tmp/container.raw', saving as 'container-raw'.
[ 8.182377] systemd-importd[1144]: Got invalid percent value '0', ignoring.
[ 8.182451] machinectl[1143]: Imported 0%.
[ 8.282669] systemd-importd[1144]: Got invalid percent value '40', ignoring.
[ 8.282746] machinectl[1143]: Imported 40%.
[ 8.366448] machinectl[1143]: Wrote 64.0M.
[ 8.366519] machinectl[1143]: Operation completed successfully.
[ 8.366617] machinectl[1143]: Exiting.
Yu Watanabe
Lennart Poettering
Adrian Vovk
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
Mike Yuan
Daan De Meyer
Antonio Alvarez Feijoo
Frantisek Sumsal