The release file that accompanies the confext images needs to be
host compatible to be able to be merged into the host /etc/ directory.
This commit checks for version compatibility between the image file and
the host file.
Adds a new image type called IMAGE_CONFEXT which is similar to IMAGE_SYSEXT but works
for the /etc/ directory instead of /usr/ and /opt/. This commit also adds the ability to
parse the release file that is present with the confext image in /etc/confext-release.d/
directory.
This is useful to identify log messages with metadata from the images
they run on. Look for ID/VERSION_ID/IMAGE_ID/IMAGE_VERSION/BUILD_ID,
with a SYSEXT_ prefix if we are looking at an extension, and append via
LogExtraFields= as respectively PORTABLE_NAME_AND_VERSION= in case of a
single image. In case of extensions, append as PORTABLE_ROOT_NAME_AND_VERSION=
for the base and one PORTABLE_EXTENSION_AND_VERSION= for each extension.
Example with a base and two extensions, with the unit coming from the
first extension:
[Service]
RootImage=/home/bluca/git/systemd/base.raw
Environment=PORTABLE=app0.raw
BindReadOnlyPaths=/etc/os-release:/run/host/os-release
LogExtraFields=PORTABLE=app0.raw
Environment=PORTABLE_ROOT=base.raw
LogExtraFields=PORTABLE_ROOT=base.raw
LogExtraFields=PORTABLE_ROOT_NAME_AND_VERSION=debian_10
ExtensionImages=/home/bluca/git/systemd/app0.raw
LogExtraFields=PORTABLE_EXTENSION=app0.raw
LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_0
ExtensionImages=/home/bluca/git/systemd/app1.raw
LogExtraFields=PORTABLE_EXTENSION=app1.raw
LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_1
When a portable service uses extensions, we use the 'main' image name
(the one where the unit was found in) as PORTABLE=. It is useful to
also list all the images actually used at runtime, as they might
contain libraries and so on.
Use PORTABLE_ROOT= for the image/directory that is used as RootImage=
or RootDirectory=, and PORTABLE_EXTENSION= for the image/directory that
is used as ExtensionImages= or ExtensionDirectories=.
Note that these new fields are only added if extensions are used,
there's no change for single-DDI portables.
Example with a base and two extensions, with the unit coming from the
first extension:
[Service]
RootImage=/home/bluca/git/systemd/base.raw
Environment=PORTABLE=app0.raw
BindReadOnlyPaths=/etc/os-release:/run/host/os-release
LogExtraFields=PORTABLE=app0.raw
LogExtraFields=PORTABLE_ROOT=base.raw
ExtensionImages=/home/bluca/git/systemd/app0.raw
LogExtraFields=PORTABLE_EXTENSION=app0.raw
ExtensionImages=/home/bluca/git/systemd/app1.raw
LogExtraFields=PORTABLE_EXTENSION=app1.raw
take_fdopen_unlocked invalidates the FD in the PortableMetadata object,
so it cannot be used later. Use parse_env_file_fd instead which is non
destructive.
Chasing symlinks is a core function that's used in a lot of places
so it deservers a less verbose names so let's rename it to chase()
and chaseat().
We also slightly change the pattern used for the chaseat() helpers
so we get chase_and_openat() and similar.
This is a combination of fdopen() and fd_reopen(). i.e. it first reopens
the fd, and then converts that into a FILE*.
We do this at various places already manually. let's move this into a
helper call of its own.
In various tools and services we have a per-system and per-user concept.
So far we sometimes used a boolean indicating whether we are in system
mode, or a reversed boolean indicating whether we are in user mode, or
the LookupScope enum used by the lookup path logic.
Let's address that, in introduce a common enum for this, we can use all
across the board.
This is mostly just search/replace, no actual code changes.
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
Curently, these two flags were implied by dissect_loop_device(), but
that's not right, because this means systemd-gpt-auto-generator will
dissect the root block device with these flags set and that's not
desirable: the generator should not cause the partition devices to be
created (we don't intend to use them right-away after all, but expect
udev to find/probe them first, and then mount them though .mount units).
And there's no point in opening the partition devices, since we do not
intend to mount them via fds either.
Hence, rework this: instead of implying the flags, specify them
explicitly.
While we are at it, let's also rename the flags to make them more
descriptive:
DISSECT_IMAGE_MANAGE_PARTITION_DEVICES becomes
DISSECT_IMAGE_ADD_PARTITION_DEVICES, since that's really all this does:
add the partition devices via BLKPG.
DISSECT_IMAGE_OPEN_PARTITION_DEVICES becomes
DISSECT_IMAGE_PIN_PARTITION_DEVICES, since we not only open the devices,
but keep the devices open continously (i.e. we "pin" them).
Also, drop the DISSECT_IMAGE_BLOCK_DEVICE combination flag, since it is
misleading, i.e. it suggests it was appropriate to specify on all
dissected blocking devices, but that's precisely not the case, see the
systemd-gpt-auto-generator case. My guess is that the confusion around
this was actually the cause for this bug we are addressing here.
Fixes: #25528
When attaching and /etc/systemd/system.attached can't be created or used
(eg: dead symlink) the logs are pretty much useless as even at debug
level there's no indication of what is going wrong.
Add some debug logs, and return a more specific error string over D-Bus.
The name "def.h" originates from before the rule of "no needless abbreviations"
was established. Let's rename the file to clarify that it contains a collection
of various semi-related constants.
The man page says nothing about "e". Glibc clearly accepts it without fuss, but
it is meaningless for a memory object (and probably doesn't work). This use is
not portable, so let's avoid it.
Allows to skip check that ensures units must not be running.
I have a use case that would use reattach, except the orchestrator
is using a non-standard versioning scheme, so image matching cannot
work. As a workaround, need to be able to detach and then attach
manually, without stopping the units to avoid extended downtimes
and loss of FD store.
The loading of an extension image from a symlink "NAME.raw" to
"NAME-VERSION.raw" failed because the release file name check worked
with the backing file of the loop device which already resolves the
symlink and thus the found name "NAME-VERSION" mismatched "NAME".
Pass the original filename and use it instead of the backing file
when available. This fixes the loading of "NAME.raw" extensions which
are a symlink to "NAME-VERSION.raw" as, e.g., may be the case when
systemd-sysupdate manages multiple versions.
Fixes https://github.com/systemd/systemd/issues/24293
Let's rework how we lock loopback block devices in two ways:
1. Lock a separate fd, instead of the main block device fd. We already
did that for our internal locking when allocating loopback block
devices, but do so for the exposed locking (i.e.
loop_device_flock()), too, so that the lock is independent of the
main fd we actually use of IO.
2. Instead of locking the device during allocation of the loopback
device, then unlocking it (which will make udev run), and then
re-locking things if we need, let's instead just keep the lock the
whole time, to make things a bit safer and faster, and not have to
wait for udev at all. This is done by adding a "lock_op" parameter to
loop device allocation functions that declares the initial state of
the lock, and is one of LOCK_UN/LOCK_SH/LOCK_EX. This change also
shortens a lot of code, since we allocate + immediately lock loopback
devices pretty much everywhere.
prefix_roota() is something we should stop using. It is bad for three
reasons:
1. As it names suggests it's supposed to be used when working relative
to some root directory, but given it doesn't follow symlinks (and
instead just stupidly joins paths) it is not a good choice for that.
2. More often than not it is currently used with inputs under control of
the user, and that is icky given it typically allocates memory on the
stack.
3. It's a redundant interface, where chase_symlinks() and path_join()
already exist as better, safer interfaces.
Hence, let's start moving things from prefix_roota() to path_join() for
the cases where that's appropriate.
When running on images you don't want to modify the /tmp
directory even if it's writable, and often it will just
be read-only. Set PrivateTmp=yes.
Fixes https://github.com/systemd/systemd/issues/23592
When we run `portablectl detach --enable --runtime`, then it triggers
`DisableUnitFilesWithFlags` DBus method and the main unit file is
removed, but its drop-ins are not. Hence, portable_detach() failed to
list existing portable units.
This makes the loop for listing portable units also accept drop-in
directories. So, all remaining drop-in directories are correctly
removed.
Before:
```
testsuite-29.sh[600]: + portablectl detach --now --runtime --enable /tmp/rootdir minimal-app0
portablectl[1391]: (Matching unit files with prefixes 'minimal-app0'.)
portablectl[1391]: Queued /org/freedesktop/systemd1/job/1812 to call StopUnit on portable service minimal-app0-foo.service.
portablectl[1391]: Removed "/run/systemd/system.attached/minimal-app0-foo.service".
portablectl[1391]: Queued /org/freedesktop/systemd1/job/1813 to call StopUnit on portable service minimal-app0.service.
portablectl[1391]: Removed "/run/systemd/system.attached/minimal-app0.service".
portablectl[1391]: Got result done/Success for job minimal-app0-foo.service
portablectl[1391]: Got result done/Success for job minimal-app0.service
portablectl[1391]: DetachImage failed: No unit files associated with '/tmp/rootdir' found attached to the system. Image not attached?
```
After:
```
testsuite-29.sh[508]: + portablectl detach --now --runtime --enable /tmp/rootdir minimal-app0
portablectl[1076]: (Matching unit files with prefixes 'minimal-app0'.)
portablectl[1076]: Queued /org/freedesktop/systemd1/job/1946 to call StopUnit on portable service minimal-app0-foo.service.
portablectl[1076]: Removed "/run/systemd/system.attached/minimal-app0-foo.service".
portablectl[1076]: Queued /org/freedesktop/systemd1/job/1947 to call StopUnit on portable service minimal-app0.service.
portablectl[1076]: Removed "/run/systemd/system.attached/minimal-app0.service".
portablectl[1076]: Removed /run/systemd/system.attached/minimal-app0.service.d/10-profile.conf.
portablectl[1076]: Removed /run/systemd/system.attached/minimal-app0.service.d/20-portable.conf.
portablectl[1076]: Removed /run/systemd/system.attached/minimal-app0.service.d.
portablectl[1076]: Removed /run/systemd/system.attached/minimal-app0-foo.service.d/10-profile.conf.
portablectl[1076]: Removed /run/systemd/system.attached/minimal-app0-foo.service.d/20-portable.conf.
portablectl[1076]: Removed /run/systemd/system.attached/minimal-app0-foo.service.d.
portablectl[1076]: Removed /run/portables/rootdir.
portablectl[1076]: Removed /run/systemd/system.attached.
```
Yu Watanabe
maanyagoenka
Lennart Poettering
Luca Boccassi
Daan De Meyer
Jan Janssen
Zbigniew Jędrzejewski-Szmek
David Tardon
Kai Lueke
Frantisek Sumsal
gtwang01
Michael Biebl