systemd-repart needs to find mkfs.ext4 for the test.
This is located in the directory /usr/sbin on openSUSE Tumbleweed.
But since the variable ALWAYS_SET_PATH in /etc/login.defs is set to yes,
su re-initializes the $PATH variable and removes /usr/sbin.
Hence, mkfs.ext4 is not found and the test fails.
Using setpriv instead of su fixes this issue and is more appropriate to
do the switch user task from root.
[zjs: move setpriv to $BASICTOOLS and force-push to retrigger CI]
In some cases, we want to exclude a directory's contents but not
the directory itself. In other cases, we want to exclude a directory
and its contents. Let's extend the denylist logic in copy.h to support
both by changing the denylist from a set to hashmap so we can store the
deny type as the value.
We also modify the repart ExcludeFiles= option to make use of this. If
a directory to exclude ends with a "/", we'll only exclude its contents.
Otherwise, we'll exclude the full directory.
(The one case that is left unchanged is '< <(subcommand)'.)
This way, the style with no gap was already dominant. This way, the reader
immediately knows that ' < ' is a comparison operator and ' << ' is a shift.
In a few cases, replace custom EOF replacement by just EOF. There is no point
in using someting like "_EOL" unless "EOF" appears in the text.
This reverts commit e59678b2cf.
We also modify the repart integration tests to make them pass with the
changes in this commit. In short, we have to make sure every file is
owned by the user executing repart. We use tee instead of cat since it
makes that easier. This also has the benefit of improving debugability
as seeing the config file contents on stdout makes it easier to know
which test is failing.
So, i think "erofs" is probably the better, more modern alternative to
"squashfs". Many of the benefits don't matter too much to us I guess,
but there's one thing that stands out: erofs has a UUID in the
superblock, squashfs has not. Having an UUID in the superblock matters
if the file systems are used in an overlayfs stack, as overlayfs uses
the UUIDs to robustly and persistently reference inodes on layers in
case of metadata copy-up.
Since we probably want to allow such uses in overlayfs as emplyoed by
sysext (and the future syscfg) we probably should ramp up our erofs game
early on. Hence let's natively support erofs, test it, and in fact
mention it in the docs before squashfs even.
--include-partitions and --exclude-partitions now fully exclude
partitions from repart. Whenever a partition type is excluded, we
don't take any partitions of that type into account at all when
running systemd-repart.
--skip-partitions= is introduced to do what --exclude-partitions did
previously. Any skipped partitions are taken into acount when doing
size calculations, but are not yet populated.
Why do we need both concepts? Exclusion is needed so that we can
use shared repart definitions to generate bootable and non-bootable
images. When generating a non-bootable image, we use --exclude-partitions
to exclude the ESP partition. Skipping is needed so that we can
populate the root partition while skipping the ESP partition, get
the roothash of the root partition, use that to generate a UKI, and
finally populate the ESP partition with the UKI included.
To make sure rootless mode keeps working, let's run all repart
integration tests that we can without root privileges. The only ones
we need to keep running with root privileges are the tests that operate
on a block/loop device and those that use --image=.
This is useful to force off fancy unicode glyph use (i.e. use "->"
instead of "→"), which is useful in tests where locales might be
missing, and thus control via $LC_CTYPE is not reliable.
Use this in TEST-58, to ensure the output checks we do aren't confused
by missing these glyphs being unicode or not.
This commit adds a new Verity= setting to repart definition files
with two possible values: "data" and "hash".
If Verity= is set to "data", repart works as before, and populates
the partition with the content from CopyBlocks= or CopyFiles=.
If Verity= is set to "hash", repart will try to find a matching
data partition with Verity=data and equal values for CopyBlocks=
or CopyFiles=, Format= and MakeDirectories=. If a matching data
partition is found, repart will generate verity hashes for that
data partition in the verity partition. The UUID of the data
partition is set to the first 128 bits of the verity root hash. The
UUID of the hashes partition is set to the final 128 bits of the
verity root hash.
Fixes#24559
The original workaround didn't work, as `systemd-repart` kept failing
even when the `/dev/loopX` device was present:
```
[ 13.959419] H testsuite-58.sh[280]: + LOOP=/dev/loop1
[ 13.959636] H testsuite-58.sh[280]: + :
[ 13.959764] H testsuite-58.sh[280]: + test -e /dev/loop1
[ 13.959895] H testsuite-58.sh[280]: + break
[ 13.960023] H testsuite-58.sh[280]: + systemd-repart --pretty=yes --definitions=/tmp/testsuite-58-sector/ --seed=750b6cd5c4ae4012a15e7be3c29e6a47 --empty=require --dry-run=no /dev/loop1
[ 13.970538] H testsuite-58.sh[363]: Device '/dev/loop1' has no dm-crypt/dm-verity device, no need to look for underlying block device.
[ 13.970538] H testsuite-58.sh[363]: Failed to determine canonical path for '/dev/loop1': No such file or directory
[ 13.970538] H testsuite-58.sh[363]: Failed to open file or determine backing device of /dev/loop1: No such file or directory
```
It seems there exists a short time period that we cannot see the
loopback device after `losetup` is finished:
```
testsuite-58.sh[367]: ++ losetup -b 1024 -P --show -f /tmp/testsuite-58-sector-1024.img
kernel: loop1: detected capacity change from 0 to 204800
testsuite-58.sh[285]: + LOOP=/dev/loop1
testsuite-58.sh[285]: + systemd-repart --pretty=yes --definitions=/tmp/testsuite-58-sector/ --seed=750b6cd5c4ae4012a15e7be3c29e6a47 --empty=require --dry-run=no /dev/loop1
testsuite-58.sh[368]: Device '/dev/loop1' has no dm-crypt/dm-verity device, no need to look for underlying block device.
testsuite-58.sh[368]: Failed to determine canonical path for '/dev/loop1': No such file or directory
testsuite-58.sh[368]: Failed to open file or determine backing device of /dev/loop1: No such file or directory
```
The Ubuntu CI on ppc64el seems to have a issue on tmpfs, and files
may not be fsynced. See c10caebb98.
For safety, let's use /var/tmp to store disk images.
We would remove stuff only if successful, so repeated invocations would
trivially fail.
Also drop "-f", so that if we expect to remove something, it must be there.
Add 'reattach' verb to portablectl, and corresponding DBUS interface
to systemd-portabled.
Takes the same parameters as 'attach', but it will do a 'detach' (and
it will refuse to proceed if it cannot be done) first, matching on
the unversioned prefix of the new image. Eg:
portablectl reattach /tmp/foo_2.raw
will cause foo_1.raw to be detached, and foo_2.raw to be attached.
The key difference with a manual 'detach old' plus 'attach new' is that
the running units are not disturbed until after the attach completed,
and if --now is passed they are then restarted.
A 'detach' is not allowed normally if the units are running.
By using a restart-after-deploy method, 'reattach' allows for minimal
interruption of service and also for features that only work on restart
(eg: file descriptor store) to work as intended.
The DBUS interface returns two lists: first the removals from the detach
that were not immediately re-added in the attach, so that the caller
can stop the relevant units, and then the list of additions that are
either new or updates, so that the caller can restart/enable the
relevant units. portablectl already implements this with the existing
--now/--enable switches.
Daan De Meyer
Thomas Blume
Franck Bui
Zbigniew Jędrzejewski-Szmek
Lennart Poettering
Yu Watanabe
Frantisek Sumsal
Luca Boccassi