Let's make fd_verify_safe_flags() even more useful:
1. let's return the cleaned up flags (i.e. just the access mode) after
validation, hiding all the noise, such as O_NOFOLLOW, O_LARGEFILE and
similar.
2. let's add a "full" version of the call that allows passing additional
flags that are OK to be set.
It's a bit weird we allow importing/pulling/exporting images, but we
have no scheme for showing what#s already downloaded. Hence let's add
this, it's easy to add after all.
Let's downgrade log levels a bit on HTTP error codes. After all we
gracefully handle many of them, and we do generated an extra message for
the ones which are fatal anyway, hence there's no point in emphasizing
the HTTP erro message levels as we currently do.
Previously, when downloading an image, importd would first download them
into one image which it would then consider immutable (named after the
originating URL/etag), and then immediately make a copy of it (named
after the client chosen name).
This makes some sense in VM/container cases where the images are
typically mutable, and thus the original downloaded copy is of some
value.
For sysexts/confexts/portable this doesn't make much sense though, as
they are typically immutable. Hence make the concept optional.
This adds --keep-download=yes/no as a new option that controls the
above. Moreover it disables the behaviour for all image classes but
"machine". The behaviour remains enabled for "machine", for compat.
A while back we introduced image_name_is_valid() for validating image
file names. It's more liberal than hostname_is_valid() in many ways (and
allows version suffixes and such). Since importd deals in offline images
(as opposed to machined otherwise which deals in running machines),
let's hence use the right helper to validate the identifiers.
This adds "Ex" versions of all bus calls import implements, that make
two changes:
1. A "class" parameter is added that allows choosing between
machine/sysext/confext/portable images to download. Depending on the
chose class the target directory is selected differently (i.e. not
just /var/lib/machines/, but alternatively /var/lib/portables/,
/var/lib/extensions/, /var/lib/confexts/.
2. The boolean flags are replaced by a 64bit flags parameter.
The two enums are mostly the same, the former is just an extension of
the latter. Let's merge them, to simplify things. This is particularly
useful as we then can reuse this systematically as D-Bus method call
flags too, in a generic fashion that works for both imports and pulls
the same.
Pretty much just renaming of flags.
This is pretty much a 1:1 copy of the importd specific part of
machinectl.
We turn this into a separate tool, so that we can eventually make the
tool generic to also download other DDIs, not just machine images.
if we try to open file:// URLs that don't exist, we'll not get IO/timer
events about it, hence it is not sufficient to check for completion in
these events. Let's add a defer event, to deal with that.
Also, curl_multi_info_read() is a queue, make sure to handle all events
that might be queued.
This simplifies bus_verify_polkit_async() and related calls quite a bit:
1. This removes any support for authentication-by-Linux-capability. This
is ultimately a kdbus leftover: with classic AF_UNIX transports we
cannot authenticate by capabilities securely (because we cannot
acquire it from the peer without races), hence we never actually did.
Since the necessary kernel work didn't materialize in the last 10y,
and is unlikely to be added, let's just kill this context. We cannot
quite remove the caps stuff from sd-bus for API compat, but for our
polkit logic let's kill it.
2. The "good_uid" and "interactive" params are only necessary in very
few cases, hence let's move them to a new call
bus_verify_polkit_async_full() and make bus_verify_polkit_async() a
wrapper around it without those two parameters.
This also fixes a bunch of wrong uses of the "interactive" bool. The
bool makes no sense today as the ALLOW_INTERACTIVE_AUTHORIZATION field
in the D-Bus message header replaces it fully. We only need it to
implement method calls we introduced prior to that header field becoming
available in D-Bus. And it should only be used on such old method calls,
and otherwise always be set to false.
This does not change behaviour in any way. Just simplifies stuff.
Fixes: #21586
Since we parse it on the other side via parse_percent() which requires
that, otherwise we get an error:
[ 8.133131] testsuite-13.sh[649]: + machinectl import-raw /tmp/container.raw container-raw
[ 8.175035] machinectl[1143]: Enqueued transfer job 1. Press C-c to continue download in background.
[ 8.182130] machinectl[1143]: Importing '/tmp/container.raw', saving as 'container-raw'.
[ 8.182377] systemd-importd[1144]: Got invalid percent value '0', ignoring.
[ 8.182451] machinectl[1143]: Imported 0%.
[ 8.282669] systemd-importd[1144]: Got invalid percent value '40', ignoring.
[ 8.282746] machinectl[1143]: Imported 40%.
[ 8.366448] machinectl[1143]: Wrote 64.0M.
[ 8.366519] machinectl[1143]: Operation completed successfully.
[ 8.366617] machinectl[1143]: Exiting.
Sometimes it makes sense to hard kill a client if we die. Let's hence
add a third FORK_DEATHSIG flag for this purpose: FORK_DEATHSIG_SIGKILL.
To make things less confusing this also renames FORK_DEATHSIG to
FORK_DEATHSIG_SIGTERM to make clear it sends SIGTERM. We already had
FORK_DEATHSIG_SIGINT, hence this makes things nicely symmetric.
A bunch of users are switched over for FORK_DEATHSIG_SIGKILL where we
know it's safe to abort things abruptly. This should make some kernel
cases more robust, since we cannot get confused by signal masks or such.
While we are at it, also fix a bunch of bugs where we didn't take
FORK_DEATHSIG_SIGINT into account in safe_fork()
We use it for more than just pipe() arrays. For example also for
socketpair(). Hence let's give it a generic name.
Also add EBADF_TRIPLET to mirror this for things like
stdin/stdout/stderr arrays, which we use a bunch of times.
We usually check return value of syscalls or glibc functions by it is
negative or not, something like that `if (stat(path, &st) < 0)`.
Let's also use the same style for lseek() and friends even the type of
their return value is off_t.
Note, fseeko() returns int, instead of off_t.
The subvolumes set is a set of source inodes similar to how the
denylist hashmap contains source inodes as keys. It indicates
directories in the source tree that should become subvolumes in
the target tree.
The commit b640e274a7 introduced reflink()
and reflink_full(). We usually name function xyz_full() for fully
parameterized version of xyz(), and xyz() is typically a inline alias of
xyz_full(). But in this case, reflink() and reflink_full() call
different ioctl().
Moreover, reflink_full() does partial reflink, while reflink() does full
file reflink. That's super confusing.
Let's rename reflink_full() to reflink_range(), the new name is
consistent with ioctl name, and should be fine.
The kernel has had filesystem independent reflink ioctls for a
while now, let's try to use them and fall back to the btrfs specific
ones if they're not supported.
Lennart Poettering
Mike Yuan
Yu Watanabe
Frantisek Sumsal
Zbigniew Jędrzejewski-Szmek
Daan De Meyer
Luca Boccassi
Jan Janssen