This time, various commands with non-trivial requirements are handled.
So the series is shorter, but has a pattern of feature additions and
then those new features being used. The way that --help output is
formatted is changed a bit in various places. Those changes are
described in individual commits.
22 'static int help()' functions left after this.
We already have io.systemd.service.SetLogLevel. Add a getter so that
CLI tools with `log-level` subcommands can be more easily implemented
with varlink.
Bind this method to all services that already bind io.systemd.service.SetLogLevel.
Almost all programs are simple and have only a single command. When
printing options, let's just use the first COMMAND block found by default.
This means we can drop the explicit program-name argument and reduce
copy&paste boilerplate.
Previously, the userdata assigned varlink operation was Manager object or
SyncReq object, depends on if synchronization operation is queued or not.
When there is a method that sets a custom userdata, we should not
inherit the userdata of sd_varlink_server to sd_varlink.
Let's drop SD_VARLINK_SERVER_INHERIT_USERDATA flag from the server,
and always obtain the Manager object from the server.
Follow-up for c2806a8c1d.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added. The description refers to the "journalctl"
option namespace; the varlink-mode namespace is not described.
The custom help() is replaced by a COMMAND() description, and
--introspect-cli is added. "COMMAND…" is marked as optional, since
'cat' is the default verb.
Since 4c0d8d9673, most dlopen notes are
set at the beginning of the executables. Let's manage all dlopen notes
there, rather than setting them where dlopen is called.
Note that the only exceptions are the LIBBPF_NOTE for networkd and
nsresource. Since dlopen_bpf() is wrapped in an `#if` guard, the notes
are instead set within the corresponding functionality.
As a result, the DLOPEN_FOO() wrapper macros are no longer needed and
can be dropped completely.
Reproducer:
journalctl -F _SYSTEMD_UNIT -u test.service --no-pager
journalctl -u test.service --no-pager -n 1
Before, the field listing ignored the unit filter and listed unrelated
units, while the normal journal query applied the filter.
sd_journal_query_unique() cannot represent journalctl filters such as
unit, boot, time, cursor, or grep filters. Walking the journal line by
line would make field listing linear in the number of entries and
duplicate unique-value handling.
Keep the scope-option predicate next to add_filters(), and reject field
listings combined with options that actually limit the journal.
Display-only options such as --pager-end are left alone.
Previously, even though sd-journal does not support bzip2 and gzip
compression, the dlopen ELF notes for those libraries were still being
attached to libsystemd.so and various journal-handling executables.
This occurred because the unified compression/decompression interfaces
handled all formats unconditionally, causing the compiler and linker to
pull in all associated dlopen notes across the board.
To resolve this, split these functions into generic and journal-specific
variants (e.g., introducing compress_blob_journal() and decompress_blob_journal()).
The journal-specific variants only handle formats actually supported by
the journal (LZ4, XZ, and ZSTD).
By updating sd-journal, journald, journalctl, and related utilities to
use these new `_journal` interfaces and switching them to the narrower
COMPRESS_JOURNAL_NOTE macro, we ensure that unnecessary dlopen notes
(for bzip2 and gzip) are no longer embedded into these binaries.
Previously, when a library was dynamically loaded via a helper function
inside libsystemd-shared.so, the resulting dlopen ELF note was not
propagated to the invoking executable's ELF metadata.
This commit explicitly annotates each executable with the relevant
dlopen ELF notes for any optional dependencies it might potentially
load. This ensures that package managers and build systems can properly
discover runtime dependencies that are triggered indirectly.
This also switches all these notes to be defined via SD_ELF_NOTE_DLOPEN_ANCHORED().
Consequently, any notes added within unreachable or unused functions will be
automatically garbage-collected by the linker (--gc-sections) instead of
bloating the final binary.
E.g. unused p11-kit library dependency is now dropped from
systemd-repart.standalone binary.
Before:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
idn Support for internationalized domain names libidn2.so.0 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
p11-kit Support for PKCS11 hardware tokens libp11-kit.so.0 suggested
```
After:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
```
Extend the 'install' keyword for executable definitions to accept four
modes—'yes', 'no', 'both', and 'static'—to elegantly manage the creation
and installation of both shared and statically-linked (.standalone) binaries.
- 'yes' / 'no': Standard behavior (mapped to true/false).
- 'both': Installs both the shared and static variants.
- 'static': Installs the static variant under the original name, while
suffixing the uninstalled shared variant with '.shared'.
With this change, any arbitrary executable can now have its `.standalone`
variant built on demand simply by invoking `ninja <target>.standalone`.
For example, `varlinkctl` did not previously support a standalone variant,
but it can now be built explicitly via `ninja varlinkctl.standalone`.
These `.standalone` binaries are not built by default unless explicitly
specified as a ninja target or enabled via `-Dstandalone-binaries=true`.
Thus, the default build time should remain unaffected.
This centralisation eliminates a massive amount of boilerplate and duplicated
target declarations across almost all subdirectories (e.g., systemd-repart,
systemd-tmpfiles, systemd-shutdown, and systemd-report tools).
manager_open_syslog_socket() may be called again with an already open
syslog fd. In that case the old syslog event source still watches the
fd, so adding a new IO source for the same fd fails with -EEXIST and
leaves the old source installed.
Disable and unref the old event source before adding the replacement.
This keeps reopening idempotent and avoids leaving a stale event source
around.
Follow-up for f9a810beda
manager_process_native_message() looks up the sender's ClientContext with
client_context_get(), which does not pin it.
A native message may carry an OBJECT_PID=, which makes
manager_dispatch_message_real() perform a nested client_context_get() for that
PID. On a cache miss this runs client_context_try_shrink_to(), whose pid-flush
branch frees every unpinned cache entry whose PID has already been reaped,
including the borrowed sender context. manager_dispatch_message_real() then keeps
using it (it reads c->uid, and the built iovecs still alias the context's
fields).
Pin the sender context with client_context_acquire() for the duration of the
dispatch and release it afterwards, mirroring what the stdout stream path
already does.
Follow-up for 22e3a02b9d
We have two concepts: a list of files that should be extracted from an
intermediate build artifact to be used in a later build artifact, and for a
given artifact, a list "donors". The first list was called "extract", because
it is passed to the .extract_objects() meson function, and the second was
called "objects", because that is the meson parameter to specify pre-built
objects files (6350d2dbd9). But this naming is
confusing: we don't care about the 'extract' step, this is something internal
to the build machinery. And 'objects' is a very generic term.
Let's use 'export' for the stuff that is "exported" for other binaries to use,
and 'import' to say where to import from. Those terms are symmetrical and the
association between them should be intuitive. (If you think the terms are
actually assymetrical, there are precedents for confusing the import with the
import sources. E.g. in modern English, turkeys are called so because they were
imported from the Americas and guineafowls were imported from Africa via
Türkiye and all that foreign stuff is alike.)
This commit implements `journalctl -f` like behavior for the
varlink API of journalctl. It is used via:
```
$ varlinkctl call -E \
/run/systemd/io.systemd.JournalAccess \
io.systemd.JournalAccess.GetEntries '{"follow": true, "limit": 10}'
```
This gives the last 10 message and then it keeps the connection
open and output each new log line that matches the set filters.
The code is modeled after `journalctl -f`. It seems there is little
to extract into shared code here so I left it for now.
This makes each `foo_cflags` dependency define a `SYSTEMD_CFLAGS_MARKER_FOO`
macro, and checks if the macro is set when headers provided by external
libraries are included.
With this, we can fail fast at compile time if necessary `_cflags`
dependencies are omitted in meson.build. Missing dependencies found by this
mechanism have been added across the tree.
Overriding <linux/audit.h> to implicitly include <libaudit.h> turns out to
be problematic. Because <linux/audit.h> is pulled in by various other kernel
headers, any source file including those kernel headers indirectly ends up
depending on <libaudit.h>, even if the executable or library being built
does not require libaudit at all.
Let's drop the override header and include <libaudit.h> only where explicitly
needed. This also moves the fallback definitions of AUDIT_SERVICE_* and
MAX_AUDIT_MESSAGE_LENGTH to libaudit-util.h.
This introduce a vtable for journal tagging feature in sd-journal,
and makes libgcrypt dependent features loaded by users (journald,
journalctl, journal-remote, and unit tests) when necessary.
Then, move several components for journal tagging in JournalFile
to JournalAuthContext.
This also introduces wrapper functions that checks gcrypt support.
client_context_read_extra_fields() reads a 64-bit field length v from
the per-unit log-extra-fields file. n = sizeof(uint64_t) + v overflows
when v is near UINT64_MAX, so the "left < n" check is bypassed and the
following memchr() scans v bytes past the buffer. Bound v against the
remaining bytes instead, which cannot overflow.
This commit exposes the last 10 high priority logs as metrics
so that the systemd-report reports them. The entries are
reported as `io.systemd.Journal.HighPriorityMessage` and
include all field as the new METRIC_FAMILY_TYPE_OBJECT.
Individual fields from a journal entry that are unprintable
(invalid utf-8) are skipped.
This is archived via a new socket-activated unit listens on
/run/systemd/report/io.systemd.Journal
journalctl needs gcrypt to set up the journal sealing
keys and other such operations, but gets no dlopen note
with the dependency. Add the dlopen macro with a recommends
level so it can be skipped, but gets pulled in by default
on user/desktop systems.
Follow-up for 590e226437
Our baseline glibc is 2.34, which merged libdl, libpthread (the
dependency('threads') target), and librt into libc. Empty .so/.a stubs
remain for backward compatibility with old binaries, but new builds
resolve dl_*, pthread_*, mq_*, timer_*, etc. directly from libc.
On musl the same libraries are likewise empty stubs.
Drop the libdl, threads, and librt entries from every meson.build, and
remove the now-stale 'Libs.private: -lrt -pthread' from libudev.pc.in
since both flags resolve to empty link-time stubs on glibc 2.34+ and
musl.
Verified with readelf -d that libsystemd.so, libudev.so, and systemd no
longer carry DT_NEEDED entries for libdl/libpthread/librt.
The help string for --smart-relinquish-var and --relinquish-var
were in reversed order because of the _fallthrough_.
We would resolve the conditions for "smart relinquish" immediately
in parse_argv() and call 'return 0' if the conditions were wrong,
terminating option parsing and the program. It seems nicer to delay
action until later. This makes the logic flow more standard. This
also allows the option parsing cases to be exchanged, fixing the
issue with --help.
Two namespaces are used: "journalctl" and "journalctl-varlink". Help for
--user/--system in the latter is added, even though it is not used yet.
I think it'll be good to have this for introspection.
The four FSS-related options (--interval, --verify-key, --force,
--setup-keys) unfortunately each gain an inline #if HAVE_GCRYPT / #else;
the EOPNOTSUPP fallback is duplicated four times.
The metavar for --identifier/--exclude-identifier is changed to "ID"
to make the layout nicer. (And because that seems to make more sense.)
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
Pure reordering. ARG_SMART_RELINQUISH_VAR is kept immediately before
ARG_RELINQUISH_VAR because of the existing _fallthrough_; that's the
only deviation from strict --help order.
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
felix h
Yu Watanabe
Zbigniew Jędrzejewski-Szmek
Nick Rosbrook
dongshengyuan
Lennart Poettering
Luca Boccassi
Michael Vogt
Syed Mohammed Nayyar
Daan De Meyer