When compiler optimization or LTO (Link Time Optimization) is enabled,
dlopen helper functions (such as dlopen_libfoo()) can be aggressively
inlined into their callers or cloned for specific call sites.
While this is beneficial for production builds, it alters or completely
removes the original function symbols from the final executable.
Consequently, this breaks the test (to be added in this series) that
checks whether the dlopen_libfoo() function corresponding to a dlopen
ELF note is actually called.
To resolve this, introduce the `_dlopen_loader_` macro. Under developer
builds, it applies the `_noclone_` and `_noinline_` attributes to guarantee
that helper symbols remain intact and discoverable by the test suite. In
production builds, the macro evaluates to nothing, allowing full compiler
optimization to proceed unimpeded.
This also switches all these notes to be defined via SD_ELF_NOTE_DLOPEN_ANCHORED().
Consequently, any notes added within unreachable or unused functions will be
automatically garbage-collected by the linker (--gc-sections) instead of
bloating the final binary.
E.g. unused p11-kit library dependency is now dropped from
systemd-repart.standalone binary.
Before:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
idn Support for internationalized domain names libidn2.so.0 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
p11-kit Support for PKCS11 hardware tokens libp11-kit.so.0 suggested
```
After:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
```
This makes each `foo_cflags` dependency define a `SYSTEMD_CFLAGS_MARKER_FOO`
macro, and checks if the macro is set when headers provided by external
libraries are included.
With this, we can fail fast at compile time if necessary `_cflags`
dependencies are omitted in meson.build. Missing dependencies found by this
mechanism have been added across the tree.
Each dlopen_*() wrapper kept its dl handle as a file-scope
'static void *xxx_dl = NULL;' even though only the wrapper itself
ever referenced it. Move each one inside the corresponding function
so its scope matches its actual use, leaving the rest of each
translation unit free of the unused file-scope name.
In pcre2-util.c the assert(pcre2_dl) in pattern_matches_and_log()
becomes assert(sym_pcre2_match), which carries the same invariant
(pattern_compile_and_log() ran dlopen_pcre2()).
For libc syscall wrappers (pidfd_open, fsopen, openat2, etc.) we previously
gated the calls behind build-time HAVE_* checks. Replace these with shim
functions in src/libc/ that fall back to the raw syscall at runtime when the
loaded glibc lacks the symbol. The infrastructure lives in src/libc/libc-shim.h:
DEFINE_SYSCALL_SHIM falls back to a direct syscall, DEFINE_LIBC_SHIM returns
ENOSYS (for posix_spawn-family helpers that have no corresponding syscall), and
DEFINE_LIBC_ERRNO_SHIM sets errno=ENOSYS and returns -1 (for read/write-style
helpers). The weak reference to the libc symbol is bound via __asm__(\"name\")
rename so the bare libc identifier never appears as a C token — this avoids
both #undef boilerplate against override-header redirects and the resulting
-Wredundant-decls warning. Drop the corresponding cc.has_function() loop from
meson.build.
For optional libraries (libcryptsetup, libdw, libarchive), drop the per-symbol
HAVE_* checks. Always declare the prototypes, suppressing the redundant-decl
warnings via DISABLE_WARNING_REDUNDANT_DECLS and NOLINT, and resolve the symbols
after the main dlopen via a new DLSYM_OPTIONAL() helper that only assigns on
success. libcryptsetup's crypt_set_keyring_to_link / crypt_token_set_external_path
and libarchive's *_is_set wrappers use fallback functions as their pointer
initializers (returning -ENOSYS and 0 respectively), so call sites can invoke
the symbol unconditionally and just check for -ENOSYS where the \"not supported\"
distinction matters.
The same shim treatment applies to pidfd_spawn / posix_spawnattr_setcgroup_np
(src/libc/spawn.c) and epoll_pwait2 (src/libc/epoll.c), with corresponding
override headers in src/include/override/spawn.h and
src/include/override/sys/epoll.h. posix_spawn_wrapper() in process-util.c and
epoll_wait_usec() in sd-event.c now detect ENOSYS in the return value instead
of checking the function pointer, falling back to plain posix_spawn() and
epoll_wait() respectively. coredump-config and coredump-submit get a
dlopen_dw_has_dwfl_set_sysroot() helper. The kexec arch gate now uses
defined(__NR_kexec_file_load) directly; pidfd.h uses __has_include_next() to
decide whether to pull in glibc's header.
This lets binaries built against newer glibc / libcryptsetup / libdw /
libarchive headers still load and run on older targets where these symbols are
absent.
Every dlopen_xxx() helper now takes an int log_level argument. It is
passed through to dlopen_many_sym_or_warn() (which in turn propagates it
to dlopen_verbose() for the library-not-installed case), and is used by
the fallback stub when support for the library is not compiled in to
emit a "<lib> support is not compiled in." message at the caller's level.
Callers pass LOG_DEBUG when gracefully degrading, or a higher level when
the failure should surface, and no longer need to log redundantly at the
call site.
As part of this, dlopen_bpf_full() (which already took a log_level) is
merged into dlopen_bpf() rather than keeping both.
The static inline fallbacks used to live in the headers, which required
pulling log.h in from every header that declared a dlopen_xxx(). Move
them into the .c files instead: the declaration is always outside the
#if HAVE_XXX block, the impl sits outside the outer #if HAVE_XXX wrap
with its own internal #if HAVE_XXX/#else/#endif, and apparmor-util.c,
idn-util.c, libmount-util.c and pam-util.c are now always compiled so
they can host their stubs.
Switch all internal callers from the private ELF_NOTE_DLOPEN() macro to
the new public SD_ELF_NOTE_DLOPEN() API from sd-dlopen.h, and remove
the now-redundant macro definitions from dlfcn-util.h.
Signed-off-by: Christian Brauner <brauner@kernel.org>
Major distributions already have elfutils >= 0.190.
Let's bump the required minimum version.
Note, elfutils 0.177 was released on 2019-08-14.
See also #38608.
Now that the necessary functions from log.h have been moved to macro.h,
we can stop including log.h in macro.h. This requires modifying source
files all over the tree to include log.h instead.
read() returns ssize_t (i.e. 64bit typically). We assigned it to int
variables in some cases (i.e. 32bit typically). Let's not be so sloppy,
and not accidentally drop 32bit on the floor.
(of course, this is not an issue IRL since we'll not have allocations
above 2^32 ever we could read into, but still, let's clean this up)
This is preparation for making our Varlink API a public API. Since our
Varlink API is built on top of our JSON API we need to make that public
first (it's a nice API, but JSON APIs there are already enough, this is
purely about the Varlink angle).
I made most of the json.h APIs public, and just placed them in
sd-json.h. Sometimes I wasn't so sure however, since the underlying data
structures would have to be made public too. If in doubt I didn#t risk
it, and moved the relevant API to src/libsystemd/sd-json/json-util.h
instead (without any sd_* symbol prefixes).
This is mostly a giant search/replace patch.
When looking at how dlopen for various libs is implemented, I found that the
macros hide too much. I find it much easier to see what is going on if 'extern'
and '= NULL' are written explicitly. After all, we don't hide those for other
definitions, e.g. our style guide says that static variables should be
initialized with '= NULL'. With that change, it's much more obvious what is
a variable declaration and what is a variable initialization.
Use 'recommended' priority for the default compression library, to
indicate that it should be prioritized over the other ones, as it
will be used to compress journals/core files.
Also use 'recommended' for kmod, as systems will likely fail to boot
if it's missing from the initrd.
Use 'suggested' for everything else.
There is one dlopen'ed TPM library that has the name generated
at runtime (depending on the driver), so that cannot be added, as it
needs to be known at build time.
Also when we support multiple ABI versions list them all, as for the
same reason we cannot know which one will be used at build time.
$ dlopen-notes.py build/libsystemd.so.0.39.0 build/src/shared/libsystemd-shared-256.so
libarchive.so.13 suggested
libbpf.so.0 suggested
libbpf.so.1 suggested
libcryptsetup.so.12 suggested
libdw.so.1 suggested
libelf.so.1 suggested
libfido2.so.1 suggested
libgcrypt.so.20 suggested
libidn2.so.0 suggested
libip4tc.so.2 suggested
libkmod.so.2 recommended
liblz4.so.1 suggested
liblzma.so.5 suggested
libp11-kit.so.0 suggested
libpcre2-8.so.0 suggested
libpwquality.so.1 suggested
libqrencode.so.3 suggested
libqrencode.so.4 suggested
libtss2-esys.so.0 suggested
libtss2-mu.so.0 suggested
libtss2-rc.so.0 suggested
libzstd.so.1 recommended
Co-authored-by: Luca Boccassi <bluca@debian.org>
We have these pretty macros, let's use them everywhere (so far we mostly
used them for newer additions only).
This PR is mostly an excercise in "perl -p -i -e", but there are some
special cases:
* idn-util.c exposes a function whose prototype in the official library
headers is marked with the "const" attribute, and this apparently does
not propagate along typeof() correctly and then
__builtin_types_compatible_p() fails later because it detects that
prototype and original function don't match in prototype.
* libbpf removed some symbols in newer versions, hence we need to define
some prototypes manually to still be able to build.
* libcryptsetup marked a symbol as deprecated we want to use (knowing it
is deprecated). By using the macros this is detected by the compiler.
We work around it via the usual warning off macros.
Note by using these macros we assume that all symbols are known during
build time. Which might not be the case. We might need to revert this
commit for some symbols if this trips up builds on older distros.
ADD_SAFE/SUB_SAFE/MUL_SAFE do addition/subtraction/multiplication
respectively with an overflow check. If an overflow occurs these return
false, otherwise true. Example: (c = a + b) would become ADD_SAFE(&c, a,
b)
INC_SAFE/DEC_SAFE/MUL_ASSIGN_SAFE are like above but they also reassign
the first argument. Example: (a += b) would become INC_SAFE(&a, b)
We use it for more than just pipe() arrays. For example also for
socketpair(). Hence let's give it a generic name.
Also add EBADF_TRIPLET to mirror this for things like
stdin/stdout/stderr arrays, which we use a bunch of times.
We usually check return value of syscalls or glibc functions by it is
negative or not, something like that `if (stat(path, &st) < 0)`.
Let's also use the same style for lseek() and friends even the type of
their return value is off_t.
Note, fseeko() returns int, instead of off_t.
json_append() is a useful wrapper around json_variant_merge(). However,
I think the naming sould be cleaned up a bit of both functions.
I thinker "merge" is the better word than "append", since it does
decidedly more than just append: it replaces existing fields of the same
name, hence "merge" sounds more appropriate. This is as opposed to the
similar operations for arrays, where no such override logic is applied
and we really just append, hence those functions are called "append"
already.
To make clearer that "merge" is about objects, and "append" about
arrays, also include "object" in the name.
Also, include "json_variant" in the name, like we do for almost all
other functions in the JSON API that take a JSON object as primary
input, and hence are kinda object methods.
Finally, let's follow the logic that helpers that combine json_build()
with some other operation get suffixed with "b" like we already have in
some cases.
Hence:
json_variant_merge() → json_variant_merge_object()
json_append() → json_variant_merge_objectb()
This mirrors nicely the existing:
json_variant_append_array()
json_vairant_append_arrayb()
This also drops the variant of json_append() that takes a va_arg
parameter (i.e. json_appendv()). We have no user of that so far, and
given the nature as a helper function only I don#t see that happening,
and if it happens after all it's trivial to bring back.
Indeed when iterating over all the PT_LOAD segment of the core dump
while trying to look for the elf headers of a given module, we iterate
over them all and try to use the first one for which we can parse a
package metadata, but the start address is never taken into account,
so absolutely nothing guarantees we actually parse the right ELF header
of the right module we are currently iterating on.
This was tested like this:
- Create a core dump using sleep on a fedora 37 container, with an
explicit LD_PRELOAD of a library having a valid package metadata:
podman run -t -i --rm -v $(pwd):$(pwd) -w $(pwd) fedora:37 bash -x -c \
'LD_PRELOAD=libreadline.so.8 sleep 1000 & SLEEP_PID="$!" && sleep 1 && kill -11 "${SLEEP_PID}" && mv "core.${SLEEP_PID}" the-core'
- Then from a fedora 38 container with systemd installed, the resulting
core dump has been passed to systemd-coredump with and without this
patch. Without this patch, we get:
Module /usr/bin/sleep from rpm bash-5.2.15-3.fc38.x86_64
Module /usr/lib64/libtinfo.so.6.3 from rpm coreutils-9.1-8.fc37.x86_64
Module /usr/lib64/libc.so.6 from rpm coreutils-9.1-8.fc37.x86_64
Module /usr/lib64/libreadline.so.8.2 from rpm coreutils-9.1-8.fc37.x86_64
Module /usr/lib64/ld-linux-x86-64.so.2 from rpm coreutils-9.1-8.fc37.x86_64
While with this patch we get:
Module /usr/bin/sleep from rpm bash-5.2.15-3.fc38.x86_64
Module /usr/lib64/libtinfo.so.6.3 from rpm ncurses-6.3-5.20220501.fc37.x86_64
Module /usr/lib64/libreadline.so.8.2 from rpm readline-8.2-2.fc37.x86_64
So the parsed package metadata reported by systemd-coredump when the module
files are not found on the host (ie the case of crash inside a container) are
now correct. The inconsistency of the first module in the above example
(sleep is indeed not provided by the bash package) can be ignored as it
is a consequence of how this was tested.
In addition to this, this also fixes the performance issue of
systemd-coredump in case of the crashing process uses a large number of
shared libraries and having no package metadata, as reported in
https://sourceware.org/pipermail/elfutils-devel/2023q2/006225.html.
When closing the FILE handle attached to a memstream, it may attempt to
do a realloc() that may fail during OOM situations, in which case we are
left with the buffer pointer pointing to NULL and buffer size > 0. For
example:
```
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
void *realloc(void *ptr, size_t size) {
return NULL;
}
int main(int argc, char *argv[])
{
FILE *f;
char *buf;
size_t sz = 0;
f = open_memstream(&buf, &sz);
if (!f)
return -ENOMEM;
fputs("Hello", f);
fflush(f);
printf("buf: 0x%lx, sz: %lu, errno: %d\n",
(unsigned long) buf, sz, errno);
fclose(f);
printf("buf: 0x%lx, sz: %lu, errno: %d\n",
(unsigned long) buf, sz, errno);
return 0;
}
```
```
$ gcc -o main main.c
$ ./main
buf: 0x74d4a0, sz: 5, errno: 0
buf: 0x0, sz: 5, errno: 0
```
This might do unexpected things if the underlying code expects a valid
pointer to the memstream buffer after closing the handle.
Found by Nallocfuzz.
In both cases, the json string is short, so we can print it, which is useful
for diagnosing invalid data in packages. But we need escape non-printable
characters.
https://bugzilla.redhat.com/show_bug.cgi?id=2152685
I went over the rest of the codebase, and it seems that other calls to
json_parse() don't have this problem.
We would deadlock when passing the data back from the forked-off process that
was doing backtrace generation back to the coredump parent. This is because we
fork the child and wait for it to exit. The child tries to write too much data
to the output pipe, and and after the first 64k blocks on the parent because
the pipe is full. The bug surfaced in Fedora because of a combination of four
factors:
- 87707784c7 was backported to v251.5, which
allowed coredump processing to be successful.
- 1a0281a3eb was NOT backported, so the output
was very verbose.
- Fedora has the ELF package metadata available, so a lot of output can be
generated. Most other distros just don't have the information.
- gnome-calendar crashes and has a bazillion modules and 69596 bytes of output
are generated for it.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2135778.
The code is changed to try to write data opportunistically. If we get partial
information, that is still logged. In is generally better to log partial
backtrace information than nothing at all.
Yu Watanabe
Daan De Meyer
Lennart Poettering
Ambareesh Balaji
Christian Brauner
Mike Yuan
Daan De Meyer
Jelle van der Waa
Zbigniew Jędrzejewski-Szmek
Ivan Kruglov
Michal Sekletar
Lennart Poettering
Adrian Vovk
Frantisek Sumsal
Romain Geissler