This adds "Ex" versions of all bus calls import implements, that make
two changes:
1. A "class" parameter is added that allows choosing between
machine/sysext/confext/portable images to download. Depending on the
chose class the target directory is selected differently (i.e. not
just /var/lib/machines/, but alternatively /var/lib/portables/,
/var/lib/extensions/, /var/lib/confexts/.
2. The boolean flags are replaced by a 64bit flags parameter.
Since we parse it on the other side via parse_percent() which requires
that, otherwise we get an error:
[ 8.133131] testsuite-13.sh[649]: + machinectl import-raw /tmp/container.raw container-raw
[ 8.175035] machinectl[1143]: Enqueued transfer job 1. Press C-c to continue download in background.
[ 8.182130] machinectl[1143]: Importing '/tmp/container.raw', saving as 'container-raw'.
[ 8.182377] systemd-importd[1144]: Got invalid percent value '0', ignoring.
[ 8.182451] machinectl[1143]: Imported 0%.
[ 8.282669] systemd-importd[1144]: Got invalid percent value '40', ignoring.
[ 8.282746] machinectl[1143]: Imported 40%.
[ 8.366448] machinectl[1143]: Wrote 64.0M.
[ 8.366519] machinectl[1143]: Operation completed successfully.
[ 8.366617] machinectl[1143]: Exiting.
We usually check return value of syscalls or glibc functions by it is
negative or not, something like that `if (stat(path, &st) < 0)`.
Let's also use the same style for lseek() and friends even the type of
their return value is off_t.
Note, fseeko() returns int, instead of off_t.
The kernel has had filesystem independent reflink ioctls for a
while now, let's try to use them and fall back to the btrfs specific
ones if they're not supported.
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
We have free_and_replace() and friends, they are all named with lower
letters, even they are macros, not functions.
For consistency, let's rename CLOSE_AND_REPLACE() with lower letters.
This also mekes the macro used more places.
Previously the mkdir_label() family of calls was implemented in
src/shared/mkdir-label.c but its functions partly declared ins
src/shared/label.h and partly in src/basic/mkdir.h (!!). That's weird
(and wrong).
Let's clean this up, and add a proper mkdir-label.h matching the .c
file.
Only if we create a file we know for sure that it is empty and hence our
sparse file logic of skipping over NUL bytes can can work. If we hwoever
are called to write data to some existing file/block device, we must do
regular writes to override everything that might be in place before.
Hence, conditionalize sparse file writing on the write offset not being
configured (which is how we internally distinguish write to existing
file and write to new file)
This reworks/modernizes the tar/raw import logic and adds the following
new features:
- Adds the ability to control btrfs subvol and quota behaviour which was
previously always on via an env var and cmdline arg
- Adds control whether to sync() stuff after writing it, similar via env
var + cmdline arg
- Similar, the QCOW2 unpacking logic that was previously the implied
default may now be controlled via env var + cmdline arg.
- adds a "direct" mode. In this mode, the systemd-import tool can be
used as a simple tool for decompressing/unpacking/installing arbitrary
files, without all the additional meta data and auxiliary resources,
i.e. outside of the immediate disk image context. Via the new
--offset= and --size-max= switches the downloaded data can be written
to specific locations of a file (which is particularly useful to use
the tool to download fs images and write them to a partition location
before actually creating the partition).
We'll later use the latter feature for "sysupdate" concept, where images
can be directly be written to partitions. That way the systemd-import
binary will be used as backend for both "systemd-importd" and
"systemd-sysupdate" and share most of the same code.
When the flag COPY_ALL_XATTRS is set, it causes the complete set of xattrs
to be copied. If the flag is unset, only xattrs from the "user" namespace
are copied.
Fixes#17178.
This merges the two flags that are passed to the ImportTar/ImportRaw
objects into a single flags parameter, which we then can extend more
easily later on.
No change in behaviour.
This is inspired by 133b34f69a which does
the same for PullTar/PullRaw.
Let's clean up hostname_is_valid() a bit: let's turn the second boolean
argument into a more explanatory flags field, and add a flag that
accepts the special name ".host" as valid. This is useful for the
container logic, where the special hostname ".host" refers to the "root
container", i.e. the host system itself, and can be specified at various
places.
let's also get rid of machine_name_is_valid(). It was just an alias,
which is confusing and even more so now that we have the flags param.
Similar to free_and_replace. I think this should be uppercase to make it
clear that this is a macro. free_and_replace should probably be uppercased
too.
"ratelimit" is a real word, so we don't need to use the other form anywhere.
We had both forms in various places, let's standarize on the shorter and more
correct one.
Let's be helpful to static analyzers which care about whether we
knowingly ignore return values. We do in these cases, since they are
usually part of error paths.
Previously, when we'd copy an individual file we'd synthesize a
user.crtime_usec xattr with the source's creation time if we can
determine it. As the creation/birth time was until recently not
queriable form userspace this effectively just propagated the same xattr
on the source to the same xattr on the destination. However, current
kernels now allow to query the birthtime using statx() and we do make
use of that now. Which means that suddenly we started synthesizing these
xattrs much more regularly.
Doing this actually does make sense, but only in very few cases:
not for the typical regular files we copy, but certainly when dealing
with disk images. Hence, let's keep this kind of propagation, but let's
make it a flag and default to off. Then turn it on whenever we deal with
disk images, and leave it off otherwise.
This is particularly relevant as overlayfs combining a real fs, and a
tmpfs on top will result in EOPNOTSUPP when it is attempted to open a
file with xattrs for writing, as tmpfs does not support xattrs, and
hence the copy-up cannot work. Hence, let's avoid synthesizing this
needlessly, to increase compat with overlayfs.
This splits out a bunch of functions from fileio.c that have to do with
temporary files. Simply to make the header files a bit shorter, and to
group things more nicely.
No code changes, just some rearranging of source files.
Let's simplify things and drop the logic that /var/lib/machines is setup
as auto-growing btrfs loopback file /var/lib/machines.raw.
THis was done in order to make quota available for machine management,
but quite frankly never really worked properly, as we couldn't grow the
file system in sync with its use properly. Moreover philosophically it's
problematic overriding the admin's choice of file system like this.
Let's hence drop this, and simplify things. Deleting code is a good
feeling.
Now that regular file systems provide project quota we could probably
add per-machine quota support based on that, hence the btrfs quota
argument is not that interesting anymore (though btrfs quota is a bit
more powerful as it allows recursive quota, i.e. that the machine pool
gets an overall quota in addition to per-machine quota).
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
When I see "test", I have to think three times what the return value
means. With "below" this is immediately clear. ratelimit_below(&limit)
sounds almost like English and is imho immediately obvious.
(I also considered ratelimit_ok, but this strongly implies that being under the
limit is somehow better. Most of the times this is true, but then we use the
ratelimit to detect triple-c-a-d, and "ok" doesn't fit so well there.)
C.f. a1bcaa07.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
This is similar to TAKE_PTR() but operates on file descriptors, and thus
assigns -1 to the fd parameter after returning it.
Removes 60 lines from our codebase. Pretty good too I think.
Lennart Poettering
Frantisek Sumsal
Yu Watanabe
Mike Yuan
Daan De Meyer
Zbigniew Jędrzejewski-Szmek
David Tardon
Andrej Lajovic
Daniel Lockyer
Daniel Mack
Michal Schmidt