An unprivileged process cannot reduce its own capability bounding set,
hence, while it is nice to reduce the set, let's not log about it
loudly, in case we are invoked unpriv (which we explicity support these
days after all).
An unpriv process also cannot detach from its netns, hence also
downgrade the warning to a debug message.
When invoked unprivileged, let's use a transiently allocated userns, so
that we can properly untar UIDs/GIDs so that the trees appear owned by
the foreign UID/GID range.
This way we have can expose identical behaviour everywhere, can make use
of our atomic replacement calls, and openat() logic, and later apply
additional tracks while unpacking, such as putting limits on UID ranges
and similar.
Let's synchronize the buffer sizes used when passing around the disk
images, i.e. size both our internal buffers and the pipe buffers the
same (so that we can always write()/read() everything in one gone -
except for the noise compression inserts).
Let's also increase the buffer sizes from 16K to 128K, which made a
difference for me, because it reduces the number of syscalls quite a
bit.
Now that the necessary functions from log.h have been moved to macro.h,
we can stop including log.h in macro.h. This requires modifying source
files all over the tree to include log.h instead.
A while back we introduced image_name_is_valid() for validating image
file names. It's more liberal than hostname_is_valid() in many ways (and
allows version suffixes and such). Since importd deals in offline images
(as opposed to machined otherwise which deals in running machines),
let's hence use the right helper to validate the identifiers.
Sometimes it makes sense to hard kill a client if we die. Let's hence
add a third FORK_DEATHSIG flag for this purpose: FORK_DEATHSIG_SIGKILL.
To make things less confusing this also renames FORK_DEATHSIG to
FORK_DEATHSIG_SIGTERM to make clear it sends SIGTERM. We already had
FORK_DEATHSIG_SIGINT, hence this makes things nicely symmetric.
A bunch of users are switched over for FORK_DEATHSIG_SIGKILL where we
know it's safe to abort things abruptly. This should make some kernel
cases more robust, since we cannot get confused by signal masks or such.
While we are at it, also fix a bunch of bugs where we didn't take
FORK_DEATHSIG_SIGINT into account in safe_fork()
We use it for more than just pipe() arrays. For example also for
socketpair(). Hence let's give it a generic name.
Also add EBADF_TRIPLET to mirror this for things like
stdin/stdout/stderr arrays, which we use a bunch of times.
Despite popular belief, the default file extracted by GNU tar is not stdin. It
is the value of the TAPE environment variable, falling back on a compile-time
constant. On my system, the default value is /dev/full, which causes tar to
just spin forever due to --ignore-zeros. Always specifying this flag is the
safe thing to do.
~$ tar --show-defaults
--format=gnu -f/dev/full -b20 --quoting-style=escape
--rmt-command=/usr/sbin/grmt
See also: ``(tar)defaults'', available via Info viewers, and in HTML form at:
https://www.gnu.org/s/tar/manual/html_node/defaults.html
The variable is not useful outside of the loop (it'll always be null
after the loop is finished), so we can declare it inline in the loop.
This saves one variable declaration and reduces the chances that somebody
tries to use the variable outside of the loop.
For consistency, 'de' is used everywhere for the var name.
rearrange_stdio() invalidates specified fds even on failure, which means
we should always invalidate the fds we pass in no matter what. Let's
make this explicit by using TAKE_FD() for that everywhere.
Note that in many places we such invalidation doesnt get us much
behaviour-wise, since we don't use the variables anymore later. But
TAKE_FD() in a way is also documentation, it encodes explicitly that the
fds are invalidated here, so I think it's a good thing to always make
this explicit here.
This reworks/modernizes the tar/raw import logic and adds the following
new features:
- Adds the ability to control btrfs subvol and quota behaviour which was
previously always on via an env var and cmdline arg
- Adds control whether to sync() stuff after writing it, similar via env
var + cmdline arg
- Similar, the QCOW2 unpacking logic that was previously the implied
default may now be controlled via env var + cmdline arg.
- adds a "direct" mode. In this mode, the systemd-import tool can be
used as a simple tool for decompressing/unpacking/installing arbitrary
files, without all the additional meta data and auxiliary resources,
i.e. outside of the immediate disk image context. Via the new
--offset= and --size-max= switches the downloaded data can be written
to specific locations of a file (which is particularly useful to use
the tool to download fs images and write them to a partition location
before actually creating the partition).
We'll later use the latter feature for "sysupdate" concept, where images
can be directly be written to partitions. That way the systemd-import
binary will be used as backend for both "systemd-importd" and
"systemd-sysupdate" and share most of the same code.
After all we want to remove the top-level dir, and make it look like the
only subdir, hence propagate the attributes from the subdir to the
top-level dir.
By default GNU tar will only read the first archive if multiple archives
are concatenated and ignore the rest. If an archive contains trailing
garbage this will hence not be recognized by tar as error, it simply
stops reading when the first archive is done (which might escalate to
SIGPIPE when invoked via a pipe).
Let's add --ignore-zeros to the tar command line when extracting. This
means:
1) if a tar archive was concatenated (i.e. generated with tar -A) we'll
process it correctly.
2) if a tar archive contains trailing garbage tar will now generate an
error message about it, instead of just throwing EPIPE, which makes
things easier to debug as broken files are not silently processed.
I think it's OK for gnu tar to ignore trailing garbage when dealing with
classic tapes drives, i.e. mediums that do not have a size limit
built-in. However, this is not what we are dealing with: we are dealing
with OS images here, that hopefully someone generated with a clean build
system, that were signed and validated and hence should not contain
trailing garbage. Hence it's better to refuse and complain thant to
silently eat up like for classic tape drives.
Fixes: #16605
This splits out a bunch of functions from fileio.c that have to do with
temporary files. Simply to make the header files a bit shorter, and to
group things more nicely.
No code changes, just some rearranging of source files.
Let's handle tarball imports nicer that have a single top-level
directory containing the OS tree: let's move everything down during
import, so that the OS tree is ready to use automatically.
Fixes: #2116
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
This is similar to TAKE_PTR() but operates on file descriptors, and thus
assigns -1 to the fd parameter after returning it.
Removes 60 lines from our codebase. Pretty good too I think.
log.h really should only include the bare minimum of other headers, as
it is really pulled into pretty much everything else and already in
itself one of the most basic pieces of code we have.
Let's hence drop inclusion of:
1. sd-id128.h because it's entirely unneeded in current log.h
2. errno.h, dito.
3. sys/signalfd.h which we can replace by a simple struct forward
declaration
4. process-util.h which was needed for getpid_cached() which we now hide
in a funciton log_emergency_level() instead, which nicely abstracts
the details away.
5. sys/socket.h which was needed for struct iovec, but a simple struct
forward declaration suffices for that too.
Ultimately this actually makes our source tree larger (since users of
the functionality above must now include it themselves, log.h won't do
that for them), but I think it helps to untangle our web of includes a
tiny bit.
(Background: I'd like to isolate the generic bits of src/basic/ enough
so that we can do a git submodule import into casync for it)
Daan De Meyer
Yu Watanabe
q66
Lennart Poettering
Arsen Arsenović
Zbigniew Jędrzejewski-Szmek