exp10() has a symbol version > 2.34 on latest glibc. To allow
dropping our baseline required glibc runtime version to <= 2.34,
let's add our own version to prevent pulling in the newer symbol
from glibc.
Our baseline glibc is 2.34, which merged libdl, libpthread (the
dependency('threads') target), and librt into libc. Empty .so/.a stubs
remain for backward compatibility with old binaries, but new builds
resolve dl_*, pthread_*, mq_*, timer_*, etc. directly from libc.
On musl the same libraries are likewise empty stubs.
Drop the libdl, threads, and librt entries from every meson.build, and
remove the now-stale 'Libs.private: -lrt -pthread' from libudev.pc.in
since both flags resolve to empty link-time stubs on glibc 2.34+ and
musl.
Verified with readelf -d that libsystemd.so, libudev.so, and systemd no
longer carry DT_NEEDED entries for libdl/libpthread/librt.
Move the push-based streaming compression API from import-compress.c
into compress.c and delete import-compress.c/h. This consolidates all
compression code in one place and makes all compression libraries
(liblzma, liblz4, libzstd, libz, libbz2) runtime-loaded via dlopen
instead of directly linked.
Introduce opaque Compressor/Decompressor types backed by a heap-
allocated struct defined only in compress.c, keeping all third-party
library headers out of compress.h.
Rewrite the per-codec fd-to-fd stream functions as thin wrappers around
the push API via generic compress_stream()/decompress_stream() taking a
Compression type parameter. Integrate LZ4 into this framework using the
LZ4 Frame API, eliminating all LZ4 special-casing.
Extend the Compression enum with COMPRESSION_GZIP and COMPRESSION_BZIP2
and add the corresponding blob, startswith, and stream functions for
both.
Rename the ImportCompress types and functions: ImportCompressType becomes
the existing Compression enum, ImportCompress becomes Compressor (with
Decompressor typedef), and all import_compress_*/import_uncompress_*
become compressor_*/decompressor_*. Rename dlopen_lzma() to dlopen_xz()
for consistency. Make compression_to_string() return lowercase by
default.
Add INT_MAX/UINT_MAX overflow checks for LZ4, zlib, and bzip2 blob
functions where the codec API uses narrower integer types than our
uint64_t parameters.
Migrate test-compress.c and test-compress-benchmark.c to the TEST()
macro framework, new assertion macros, and codec-generic loops instead
of per-codec duplication.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
asprintf is nice to use, but the _documented_ error return convention is
unclear:
> If memory allocation wasn't possible, or some other error occurs,
> these functions will return -1, and the contents of strp are undefined.
What exactly "undefined" means is up for debate: if it was really
undefined, the caller wouldn't be able to meaningfully clean up, because
they wouldn't know if strp is a valid pointer. So far we interpreted
"undefined" — in some parts of the code base — as "either NULL or a
valid pointer that needs to be freed", and — in other parts of the
codebase — as "always NULL". I checked glibc and musl, and they both
uncoditionally set the output pointer to NULL on failure.
There is also no information _why_ asprintf failed. It could be an
allocation error or format string error. But we just don't have this
information.
Let's add a wrapper that either returns a good string or a NULL pointer.
Since there's just one failure result, we don't need a separate return
value and an output argument and can simplify callers.
We want to check if the magic we got from statfs() is one of the magics listed
for one of the file systems in the given group. To do this, we'd iteratate over
the file system names, convert each name to an array of magics, and compare
those to the one we got. We were using gperf-generated lookup table for this,
so the string lookups were quick, but still this seems unnecessarily complex.
Let's just generate a simple lookup function, because we can:
$ src/basic/filesystem-sets.py fs-in-group
bool fs_in_group(const struct statfs *st, FilesystemGroups fs_group) {
switch (fs_group) {
case FILESYSTEM_SET_BASIC_API:
return F_TYPE_EQUAL(st->f_type, CGROUP2_SUPER_MAGIC)
|| F_TYPE_EQUAL(st->f_type, CGROUP_SUPER_MAGIC)
|| F_TYPE_EQUAL(st->f_type, DEVPTS_SUPER_MAGIC)
|| F_TYPE_EQUAL(st->f_type, MQUEUE_MAGIC)
|| F_TYPE_EQUAL(st->f_type, PROC_SUPER_MAGIC)
|| F_TYPE_EQUAL(st->f_type, SYSFS_MAGIC)
|| F_TYPE_EQUAL(st->f_type, TMPFS_MAGIC);
case FILESYSTEM_SET_ANONYMOUS:
return F_TYPE_EQUAL(st->f_type, ANON_INODE_FS_MAGIC)
|| F_TYPE_EQUAL(st->f_type, PIPEFS_MAGIC)
|| F_TYPE_EQUAL(st->f_type, SOCKFS_MAGIC);
...
We flatten the nested lookup of group=>fs=>magic into a single level.
The compiler can work its magic here to make the lookup quick.
Previously, the gperf table was the main "source of truth", and additional
information (e.g. which file system names are obsolete) was scattered in
various files. We would then parse the gperf file, using python, awk, grep, and
bash, and use the results in various ways. This is hard to understand and
maintain. Let's replace all of this with a single python script that generates
the requested outputs as appropriate.
$ diff -u <(git show @{u}:src/basic/filesystems-gperf.gperf) \
<(src/basic/filesystem-sets.py gperf)
shows that the outputs are the same except for comments.
Similarly, 'src/basic/filesystem-sets.py fs-type-to-string' and
'src/basic/filesystem-sets.py filesystem-sets' can be used to view
the generated code.
The check that the kernel doesn't define any new file system magics is
converted into a normal test. It doesn't seem necessary to fail the build
when that happens.
We went back and forth between 'prog.sh', files('prog.sh'), and
find_program('prog.sh'). We want to use files() or find_program() so that we
get a good error message if the file is missing. Behaviour of meson changed
over time, and in the past not all forms could be used in all places. For
example 0f4c4f3824 added find_program() in many
places to avoid repeated messages. But it seems that all recent meson versions
work fine with files().
find_program prints silly messages:
Program tools/make-man-index.py found: YES
(/home/zbyszek/src/systemd/tools/make-man-index.py)
Program tools/meson-render-jinja2.py found: YES
(/home/zbyszek/src/systemd/tools/meson-render-jinja2.py)
...
We know that those files will be found, they are part of the git checkout.
With files() this is gone and the meson output is easier to read.
Before 7d247d3cb8, we needed the file name
twice. But now we only need it once, so no need to overcomplicate things
by defining a variable with a single use.
When a variable is used, it's also easier to make a mistake and e.g.
accidentally reuse the variable later.
This also moves syscall tables and generators to the same directory.
Note, inclusion of asm/sgidefs.h is dropped, as it is already included
by unistd.h and sys/syscall.h.
- pass our system include directories to make generators use our libc
wrappers and latest kernel headers,
- include relevant headers in generated gperf file,
- use files() rather than find_program(), as the result of
find_program() cannot be passed to 'input' of custom_target(),
- move generate-bpf-delegate-configs.py to src/core/, as it is only used
by libcore.
[1] says:
> Since 0.60.0 the name argument is optional and defaults to the basename of
> the first output
We specify >= 0.62 as the supported version, so drop the duplicate name in all cases
where it is the same as outputs[0], i.e. almost all cases.
[1] https://mesonbuild.com/Reference-manual_functions.html#custom_target
- config.h is not necessary when generating lists, hence drop it.
- linux/audit.h and libaudit.h are included by missing_audit.h,
hence not necessary to include them explicitly.
Instead of using run-clang-tidy.py with its own scheduling, let's
just gather a list of source files ourselves and then use that to
add a unit test for each source file that runs clang-tidy on the
source file.
We also add a bit of logic to run clang-tidy on most header files
as well for extra coverage. This uncovered various header files that
were not standalone so this commit also includes fixes to make sure
the clang-tidy tests are all green.
We can also use this in a later commit to run clang-include-cleaner
on each source file in the same way.
meson by default adds the current source and build directory as include
directories. Because we structure our meson code by gathering a giant dict
of everything we want to do and then doing all the actual target generation
in the top level meson.build, this behavior does not make sense at all because
we end up adding the top level repository directory as an include directory
which is never what we want.
At the same time, let's also make sure the top level directory of the build
directory is not an include directory, by moving the version.h generation
into the src/version subdirectory and then adding the src/version subdirectory
of the build directory as an include directory instead of the top level
repository directory.
Making this change means that language servers such as clangd can't get
confused when they automatically insert an #include line and insert
"#include "src/basic/fs-util.h" instead of "#include "fs-util.h".
run_command()'s working directory is documented as undefined but it
seems to be the current source directory as when the -ftime-trace
clang option is enabled, -.json is written to src/basic/meson.build
every time meson is run.
Let's make sure the command is executed in the build directory so that
any auxiliary files are written there as well.
meson's target has a few issues:
- Runs on all source files regardless if they're included in the
build or not
- Doesn't have any dependencies on generated sources which means we
have to do a full build first before we can run clang-tidy
- Doesn't allow us to pass any extra arguments
To work around these, let's define our own clang-tidy target instead
using llvm's run-clang-tidy script. Alongside the clang-tidy target,
let's start keeping track of all generated sources which we make the
clang-tidy target depend on. We also add a new target which will only
generate source files which is useful for setting up the source tree
for running code analysis against it.
Even though these are in our tree, we should still treat them as
system includes which helps various tools (clangd, iwyu, ...) understand
that these are system includes and <> should be used instead of "".
Various functions in log.h are only used by asserts, and there's
enough assertion related stuff in macro.h to justify a separate header
which also makes it easier to avoid circular dependencies.
Let's introduce assert-util.h and an accompanying fundamental header
and move all the assertion related stuff over there. PROJECT_FILE is
moved over to macro.h.
Preparation for later commits, where path-lookup would be
moved into libsystemd.
Note that it currently includes sd-id128.h, hence shared/
seems more appropriate anyway.
Our variables for internal libraries are named 'libfoo' for the shared lib
variant, and 'libfoo_static' for the static lib variant. The only exception was
libbasic, because we didn't have a shared variant for it. But let's rename it
for consitency. This makes the build config easier to understand.
systemd-255 is failing a build with the latest kernel headers… Let's downgrade
this warning, because it's fine if there's a file system we don't know about
and it makes thing less brittle if we don't treat this as a hard error.
(I initially conditionalized this on BUILD_MODE, but I don't think we need a
hard error there either. A warning will be noticed and fixed.)
gcrypt is used only for journal sealing operations in libsystemd, so it
can be made into a dlopen dependency that is used only on demand. This
allows to reduce the footprint of libsystemd in the most common cases.
Keep systemd-pull and systemd-resolved with normal linking, as they are
executables, and usually built with OpenSSL support anyway.
Daan De Meyer
Zbigniew Jędrzejewski-Szmek
Mike Yuan
Yu Watanabe
Daan De Meyer
Zbigniew Jędrzejewski-Szmek
Matteo Croce
Michal Koutný
Lennart Poettering
Luca Boccassi