Otherwise you run into errors such as:
"""
../meson.build:2899:28: ERROR: File src/test/test-loop-util.c does not exist.
"""
when deleting a file in git without staging the deletion.
Move the push-based streaming compression API from import-compress.c
into compress.c and delete import-compress.c/h. This consolidates all
compression code in one place and makes all compression libraries
(liblzma, liblz4, libzstd, libz, libbz2) runtime-loaded via dlopen
instead of directly linked.
Introduce opaque Compressor/Decompressor types backed by a heap-
allocated struct defined only in compress.c, keeping all third-party
library headers out of compress.h.
Rewrite the per-codec fd-to-fd stream functions as thin wrappers around
the push API via generic compress_stream()/decompress_stream() taking a
Compression type parameter. Integrate LZ4 into this framework using the
LZ4 Frame API, eliminating all LZ4 special-casing.
Extend the Compression enum with COMPRESSION_GZIP and COMPRESSION_BZIP2
and add the corresponding blob, startswith, and stream functions for
both.
Rename the ImportCompress types and functions: ImportCompressType becomes
the existing Compression enum, ImportCompress becomes Compressor (with
Decompressor typedef), and all import_compress_*/import_uncompress_*
become compressor_*/decompressor_*. Rename dlopen_lzma() to dlopen_xz()
for consistency. Make compression_to_string() return lowercase by
default.
Add INT_MAX/UINT_MAX overflow checks for LZ4, zlib, and bzip2 blob
functions where the codec API uses narrower integer types than our
uint64_t parameters.
Migrate test-compress.c and test-compress-benchmark.c to the TEST()
macro framework, new assertion macros, and codec-generic loops instead
of per-codec duplication.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
In some builds (package builds, so with optimization and lto, but I
haven't been able to pin down the exact combination on options that
matters), we end up with items in the verbs array reordered. The order
matters (because of groups, but also because we have some specific order
for display), so this reordering is something that we don't want.
From what I was able to read, the compiler + linker generally keep the
order within a single translation unit, but this is more of a convention
and implementation choice than a guarantee. Add this attribute [1]. It
seems to have the desired effect in CI.
[1] https://gcc.gnu.org/onlinedocs/gcc-15.2.0/gcc/Common-Function-Attributes.html#index-no_005freorder-function-attribute
Add the needed assert changes to make the code clean
for the new check-pointer-deref script.
Note that for `_cmp()` functions its a bit of a question
if we want `assert()` or `POINTER_MAY_BE_NULL()` here. The
former is more correct but it adds a (small) runtime hit. If
we use POINTER_MAY_BE_NULL() it is compiled out but strictly
speaking it is wrong (so at least we would need a comment).
For some of the directories it makes more sense to keep them
excluded from the coccinelle check. Specifically:
- libc: compatibility, no asserts or systemd headers yet
- test: uses NUL internally to test crashes etc
The fix in 8f1751a111 made me wonder if we could automatically detect
when pointers are accessed but when this might not be safe. Systemd
is already using a lot of `assert(dst)` and this change now forces
us to use them.
So this commit (ab)uses coccinelle to flag any pointer parameter
dereference not preceded by assert(param), ASSERT_PTR(param), or an
explicit NULL check. It adds integration into meson as a new "coccinelle"
test suite (just like clang-tidy) and is run in CI. The check is not
perfect but seems a reasonable heuristic.
For this RFC commit it is scoped to a subset, it excludes 25 dirs right
now and includes around 100. About 300 warnings left. Busywork that I am
happy to do if there is agreement that it is worth it.
With this in place we would have caught the bug from 8f1751a111 in CI:
```
FAIL: check-pointer-deref.cocci found issues in systemd/src/boot:
diff -u -p systemd/src/boot/measure.c /tmp/nothing/measure.c
--- systemd/src/boot/measure.c
+++ /tmp/nothing/measure.c
@@ -312,7 +312,6 @@ EFI_STATUS tpm_log_tagged_event(
if (err != EFI_SUCCESS)
return err;
- *ret_measured = true;
return EFI_SUCCESS;
}
```
This also adds a new POINTER_MAY_BE_NULL() for the cases when the
called function will do the NULL check (like `iovec_is_set()`).
The infrastructure added in the previous commits added support for IMDS
client functionality, but didn't really to enable the logic by default
on suitable hosts.
This commit adds a generator that automatically hooks the IMDS
functionality into the boot process if it detects that the system is
running on a compliant cloud system. it enables both the imds daemon and
the client.
This service's job is to talk to a VM associated IMDS service provided
by the local Cloud. It tries to abstract the protocol differences
various IMDS implementations implement, but does *not* really try to
abstract more than a few basic fields of the actual IMDS metadata.
IMDS access is wrapped in a Varlink API that local clients can talk to.
If possible this makes use of the IMDS endpoint information that has
been added to hwdb in the preceeding commit. However, endpoint info can
also be provided via kernel command line and credentials. For debugging
purposes we also accept them via environment variables and command line
arguments.
This adds a concept of early-boot networking, just enough to be able to
talk to the IMDS service. It is minimally configurable via a kernel
cmdline option (and a build-time option): the user may choose between
"locked" and "unlocked" mode. In the former mode direct access to IMDS via
HTTPS is blocked via a prohibit route (and thus all IMDS communication
has to be done via systemd-imdsd@.service). In the latter case no such
lockdown takes place, and IMDS may be acquired both via this new service
and directly. The latter is typically a good idea for compatibility with
current systems, the former is preferable for secure installations.
Access to IMDS fields is controlled via PK.
This attribute was introduced in gcc 11, and our baseline is currently
8.4. So let's allow using _retain_ everywhere, but make it into a noop
if not supported.
Using __has_attribute was suggested, but with gcc-11.5.0-14.el9.x86_64,
__has__attribute(__retain__) is true, but we get a warning when the
attribute is actually used.
When install_subdir encounters a mkosi.tools tree with a /bin to
/usr/bin symlink it fails to copy it because it dereferences but still
treats it like a file.
Work around the Meson bug by excluding the mkosi.tools tree from
installation like mkosi.local is excluded. We anyway don't want the
tools tree end up there.
The purpose of the userns-restrict BPF-LSM program is to prevent the
transient ranges leaking to disk, so let's allow operations outside the
transient UID ranges, even if the mount is not allowlisted.
This is preparation for the next commits where we'll add support for mapping
the current user and the foreign UID range into the user namespaces provisioned
by nsresourced. Operations creating files/directories as these UIDs/GIDs should
not need the corresponding mount to be allowlisted with nsresourced.
Follow-up for a109189fab
This follows the existing practice for
systemd-journal-{upload,gatewayd}.service,
as I think allocating a full-blown user
specifically for this purpose is an overkill.
And with DynamicUser=yes we can also take
advantage of implied sandboxing.
We already have some varlink support for the journal to perform
some actions like `Rotate`. It would be nice to be able to query
the journal via varlink too so this commit adds a new varlinkctl
based journal service that exposes a single GetEntries() call
to retrieve journal entries. Basic filtering is supported and
we can expand the API as needed.
This is a separate `io.systemd.JournalControl` [1] service from the
existing `io.systemd.Journald` to decouple read and write (thanks
to Lennart for suggesting this).
This also extracts some shared helper so that we do not duplicate
code when generating the json or when adding the filters.
[1] The name mirrors the bootctl->io.systemd.BootControl naming.
This allows building with musl on glibc systems as follows:
env \
CC=musl-gcc \
CXX=musl-gcc \
CFLAGS="-idirafter /usr/include" \
CXXFLAGS="-idirafter /usr/include" \
meson setup --auto-features=disabled -Dlibc=musl musl
When sd_varlink_connect_url() gets an unknown URL we now
check if there is a `$LIBEXECDIR/varlink-bridges/$scheme`
binary and execute it (with the url as the first arguments).
This makes varlink more flexible as it provides a way to
dynamically add "bridges" in LIBEXECDIR/varlink-bridges/. This is
conceptually similar to the libvarlink `varlink --bridge` command
and allows to e.g. call varlink over http{,s} via e.g. the new
varlink-http-bridge.
With a running varlink-http-bridge [0] one can do:
```console
$ varlinkctl call http://localhost:8080/ws/sockets/io.systemd.Hostname \
io.systemd.Hostname.Describe {}
{
"Hostname" : "top",
...
```
Closes: https://github.com/systemd/systemd/issues/40640
[0] https://github.com/mvo5/varlink-http-bridge/pull/1
See the [design
doc](https://github.com/systemd/systemd/issues/38023#issue-3192567450)
by @ikruglov
This PR introduces the metrics API framework, adds some basic system
wide/per unit/service metrics, and a basic CLI. The PR is broken into
two commits as described below.
### Deviations from the original design
- Introduced top level field "object" for ease of filtering. Instead of
having `fields: { unit: "foo", unit_type: "service" }`, we now have
`object: foo.service` as the top level field.
### First commit
The first commit includes:
- Metrics API definitions
- Code to set up the varlink server
- The describe method which shows all the metrics families
- The list method which lists all the metrics
- Type definitions related to MetricFamily
- Common code to build json objects
### Second commit
The second commit adds some basic metrics, a basic CLI (systemd-report)
which
lists the metrics, and integration tests.
**System wide metrics:**
- units_by_type_total
- units_by_state_total
**Two per unit metrics:**
- unit_active_state
- unit_load_state
**A service state metric:**
- nrestarts
### Sample outputs
**units_by_type_total**:
```
{
"name" : "io.systemd.Manager.units_by_type_total",
"value" : 52,
"fields" : {
"type" : "target"
}
}
{
"name" : "io.systemd.Manager.units_by_type_total",
"value" : 82,
"fields" : {
"type" : "device"
}
}
{
"name" : "io.systemd.Manager.units_by_type_total",
"value" : 2,
"fields" : {
"type" : "automount"
}
}
```
**units_by_state_total**:
```
{
"name" : "io.systemd.Manager.units_by_state_total",
"value" : 216,
"fields" : {
"state" : "active"
}
}
{
"name" : "io.systemd.Manager.units_by_state_total",
"value" : 0,
"fields" : {
"state" : "reloading"
}
}
{
"name" : "io.systemd.Manager.units_by_state_total",
"value" : 120,
"fields" : {
"state" : "inactive"
}
}
```
**unit_active_state**:
```
{
"name" : "io.systemd.Manager.unit_active_state",
"object" : "multi-user.target",
"value" : "active"
}
{
"name" : "io.systemd.Manager.unit_active_state",
"object" : "systemd-sysusers.service",
"value" : "inactive"
}
```
**unit_load_state**:
```
{
"name" : "io.systemd.Manager.unit_load_state",
"object" : "multi-user.target",
"value" : "loaded"
}
```
**nrestarts**:
```
{
"name" : "io.systemd.Manager.nrestarts",
"object" : "user@0.service",
"value" : 0
}
{
"name" : "io.systemd.Manager.nrestarts",
"object" : "user-runtime-dir@0.service",
"value" : 0
}
```
systemdstatedir is created by tmpfiles conf (tmpfiles.d/systemd.conf.in)
already.
By shipping it, systemd prevents the effect of tmpfiles.d/var.conf.in:
q /var 0755 - - -
This should helps implementing the "Hermetic /usr" pattern.
Links: https://0pointer.net/blog/fitting-everything-together.html
This partially reverts commit e58ba80a40. Those
aliases were removed as part of the removal of sysvinit compat, but the removal
was immediately reported as a regression by users. In particular, users can
have one of those set as the default target, or use those names in Wants= or
Requires= dependencies. At least in Fedora, the removal was immediately
reverted.
When looking at our "portability and stability promise", I noticed that those
units are explicitly listed in the promise. Let's restore them upstream. They
are conditionalized under a new -Dcompat-sysv-interfaces=bool option, so that
downstreams like Arch that wasnt to strictly avoid legacy interfaces can easily
disable them. They are dropped from the docs, like we do for all deprecated
interfaces.
This does not yet support directory images properly
as systemd itself does not support unpriv directory
images properly yet.
The user profiles are a copy of the system profiles but without
DynamicUser=yes (can't be used by user managers) and without
ProtectHome=yes (this masks /home which breaks StateDirectory= which
is lcoated inside /home)
A recent commit made the standalone binaries always buildable
on demand, but as a side effect due to how 'meson install' works,
they are always built and installed by 'meson install' even
if the standalone-binaries= option is disabled.
Fix it so that 'meson install' only installs them if the
option is explicitly enabled, while still allowing
building them on demand.
Follow-up for 54492552a1
As announced by a few releases now, finally drop support for
sysvinit scripts.
Keep rc-local generator for now, as it's really a distinct
feature even though from the same era.
The current tree doesn't even compile with libidn(1) after
2c7bdaf9f1, which included
a non-existent call to check_dlopen_blocked() somehow.
Hence, it feels safe to just nuke legacy support from
our repo.
Note, this drops logging only test case for crypt_preferred_method(),
as that requires explicitly dlopen() the library. But, we should test
that make_salt() and friends automatically dlopen() it.
Daan De Meyer
Zbigniew Jędrzejewski-Szmek
Michael Vogt
Mike Yuan
Lennart Poettering
Kai Lüke
Luca Boccassi
Daan De Meyer
Jörg Behrmann
Yu Watanabe
nikstur
Zbigniew Jędrzejewski-Szmek
Yaping Li
Max Gautier