We already have checks in place during compression that limit the data
we compress, so they shouldn't decompress to anything larger than
DATA_SIZE_MAX unless they've been tampered with. Let's make this
explicit and limit all our decompress_blob() calls in journal-handling
code to that limit.
One possible scenario this should prevent is when one tries to open and
verify a journal file that contains a compression bomb in its payload:
```
$ ls -lh test.journal
-rw-rw-r--+ 1 fsumsal fsumsal 1.2M Apr 12 15:07 test.journal
$ systemd-run --user --wait --pipe -- build-local/journalctl --verify --file=$PWD/test.journal
Running as unit: run-p682422-i4875779.service
000110: Invalid hash (00000000 vs. 11e4948d73bdafdd)
000110: Invalid object contents: Bad message
File corruption detected at /home/fsumsal/repos/@systemd/systemd/test.journal:272 (of 1249896 bytes, 0%).
FAIL: /home/fsumsal/repos/@systemd/systemd/test.journal (Bad message)
Finished with result: exit-code
Main processes terminated with: code=exited, status=1/FAILURE
Service runtime: 48.051s
CPU time consumed: 47.941s
Memory peak: 8G (swap: 0B)
```
Same could be, in theory, possible with just `journalctl --file=`, but
the reproducer would be a bit more complicated (haven't tried it, yet).
Lastly, the change in journal-remote is mostly hardening, as the maximum
input size to decompress_blob() there is mandated by MHD's connection
memory limit (set to JOURNAL_SERVER_MEMORY_MAX which is 128 KiB at the
time of writing), so the possible output size there is already quite
limited (e.g. ~800 - 900 MiB for xz-compressed data).
Replace the duplicated magic byte signatures in decompressor_detect()
with a call to the new compression_detect_from_magic() helper and use a
switch statement to initialize the appropriate decompression context.
We generally assume that valid times returned by clock_gettime() are > 0
and < USEC_INFINITY. If this wouldn't hold all kinds of things would
break, because we couldn't distuingish our niche values from regular
values anymore.
Let's hence encode our assumptions in C, already to help static
analyzers and LLMs.
Inspired by: https://github.com/systemd/systemd/pull/41601#pullrequestreview-4094645891
The updated parsing_hacks.h file uncovered a bunch of extra
things that the check-pointer-deref coccinelle script flags.
This commit fixes them to make the tree check-pointer-deref clean.
Coverity flags range->n_entries - j as a potential underflow
in the memmove size calculation. Add assert(range->n_entries > 0)
before decrementing n_entries, which holds since the loop condition
guarantees j < n_entries.
CID#1548015
Follow-up for 8dcc66cefc
As mentioned in the grandfather commit, I want to use the _many
suffix for freeing of the contents of an array, so the functions
to free the array to get the suffix _array.
Add CpuUsage, MemoryUsage, IOReadBytes, IOReadOperations, and
TasksCurrent in a standalone socket-activated varlink service. These
metrics are gathered from the kernel via cgroup files and PID1's only
role is mapping unit names to cgroup paths — a separate process can
query PID1 once for that mapping and then read the cgroup files
directly, minimizing PID1 involvement.
The new systemd-report-cgroup-metrics service listens at
/run/systemd/report/io.systemd.CGroup and exposes:
- io.systemd.CGroup.CpuUsage
- io.systemd.CGroup.IOReadBytes
- io.systemd.CGroup.IOReadOperations
- io.systemd.CGroup.MemoryUsage (with type=current/available/peak)
- io.systemd.CGroup.TasksCurrent
This is spun out of #41078 and based on top of it. Will rebase once
that's merged.
Multiple callers of cg_get_keyed_attribute() follow the same pattern of
reading a single keyed attribute and then parsing it as uint64_t with
safe_atou64(). Add a helper that combines both steps.
Convert all existing single-key + uint64 call sites in cgtop, cgroup.c,
and oomd-util.c to use the new helper.
Move the push-based streaming compression API from import-compress.c
into compress.c and delete import-compress.c/h. This consolidates all
compression code in one place and makes all compression libraries
(liblzma, liblz4, libzstd, libz, libbz2) runtime-loaded via dlopen
instead of directly linked.
Introduce opaque Compressor/Decompressor types backed by a heap-
allocated struct defined only in compress.c, keeping all third-party
library headers out of compress.h.
Rewrite the per-codec fd-to-fd stream functions as thin wrappers around
the push API via generic compress_stream()/decompress_stream() taking a
Compression type parameter. Integrate LZ4 into this framework using the
LZ4 Frame API, eliminating all LZ4 special-casing.
Extend the Compression enum with COMPRESSION_GZIP and COMPRESSION_BZIP2
and add the corresponding blob, startswith, and stream functions for
both.
Rename the ImportCompress types and functions: ImportCompressType becomes
the existing Compression enum, ImportCompress becomes Compressor (with
Decompressor typedef), and all import_compress_*/import_uncompress_*
become compressor_*/decompressor_*. Rename dlopen_lzma() to dlopen_xz()
for consistency. Make compression_to_string() return lowercase by
default.
Add INT_MAX/UINT_MAX overflow checks for LZ4, zlib, and bzip2 blob
functions where the codec API uses narrower integer types than our
uint64_t parameters.
Migrate test-compress.c and test-compress-benchmark.c to the TEST()
macro framework, new assertion macros, and codec-generic loops instead
of per-codec duplication.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Core dumps are often very sparse, containing large zero-filled regions
whose actual disk usage can be significantly reduced by preserving
holes. Previously, decompress_stream() always wrote dense output,
expanding all zero regions into allocated disk blocks.
Each decompression backend (xz, lz4, zstd) now auto-detects whether the
output fd is suitable for sparse writes via a shared should_sparse()
helper. The check requires both S_ISREG (regular file) and !O_APPEND,
since O_APPEND causes write() to ignore the file position set by
lseek(), which would collapse the holes and corrupt the output. For
pipes, sockets, and append-mode files, dense writes are preserved via
loop_write_full() with USEC_INFINITY timeout, matching the original
behavior. After sparse decompression, finalize_sparse() sets the final
file size to account for any trailing holes.
This is transparent to callers — all public signatures are unchanged.
coredumpctl benefits automatically:
- coredumpctl debug: temp file in /var/tmp is now sparse
- coredumpctl dump -o file: output file is now sparse
- coredumpctl dump > file: redirected stdout is now sparse
- coredumpctl dump | ...: pipe output unchanged (dense)
- coredumpctl dump >> file: append mode, falls back to dense
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-developed-by: Codex (GPT-5) <noreply@openai.com>
When systemd-tmpfiles processes a 'z' (relabel) entry, fd_set_perms()
unconditionally calls label_fix_full() even when mode, owner, and group
already match. This causes setfilecon_raw() (SELinux) or xsetxattr() (SMACK)
to write the security label even if it is already correct, which on some
kernels updates the file's timestamps unnecessarily.
Fix this by comparing the current label with the desired label before
writing, and skipping the write when they already match. This is consistent
with how fd_set_perms() already skips chmod/chown when the values are
unchanged.
Timestamps are not guaranteed to be set by `statx()`, and their presence
should not be asserted as a proxy to judge the kernel version. In
particular, `STATX_ATIME` is omitted from the return when querying a
file on a `noatime` superblock, causing spurious errors from tmpfiles:
```console
# SYSTEMD_LOG_LEVEL=debug systemd-tmpfiles --clean
<...>
Running clean action for entry X /var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-*/tmp
statx() does not support 'STATX_ATIME' mask (running on an old kernel?)
statx(/var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-prometheus-smartctl-exporter.service-GKguQK/tmp) failed: Protocol driver not attached
statx() does not support 'STATX_ATIME' mask (running on an old kernel?)
statx(/var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-systemd-logind.service-k8j52T/tmp) failed: Protocol driver not attached
statx() does not support 'STATX_ATIME' mask (running on an old kernel?)
statx(/var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-irqbalance.service-7RJkev/tmp) failed: Protocol driver not attached
statx() does not support 'STATX_ATIME' mask (running on an old kernel?)
statx(/var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-chronyd.service-8hkO5G/tmp) failed: Protocol driver not attached
statx() does not support 'STATX_ATIME' mask (running on an old kernel?)
statx(/var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-dbus-broker.service-6P6LVl/tmp) failed: Protocol driver not attached
statx() does not support 'STATX_ATIME' mask (running on an old kernel?)
statx(/var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-nginx.service-B5HX8B/tmp) failed: Protocol driver not attached
Running clean action for entry x /var/tmp/systemd-private-94cc8a77688e497f96d5b9019e66ed6f-*
Running clean action for entry q /var/tmp
statx() does not support 'STATX_ATIME' mask (running on an old kernel?)
statx(/var/tmp) failed: Protocol driver not attached
<...>
```
Additionally, refactor `dir_cleanup()` slightly for self-consistency to
make
it evident that the `NSEC_INFINITY` transformation is correct.
Fixes#41227.
Coverity flags (uint64_t)sc * (uint64_t)ps as a potential overflow.
Use MUL_SAFE which Coverity understands via __builtin_mul_overflow.
Physical page count times page size cannot realistically overflow
uint64_t, but this makes it provable to static analyzers.
CID#1548042
Follow-up for 09bb6448ae
Coverity flags MALLOC_SIZEOF_SAFE(de) - offsetof(DirectoryEntries,
buffer) as a potential underflow when MALLOC_SIZEOF_SAFE returns 0.
After a successful malloc the return value is at least as large as
the requested size, but Coverity cannot trace this. Add an assert
to establish the lower bound.
CID#1548020
Follow-up for 6393b847f4
Coverity flags range->n_entries - j - 1 and j-- as potential
underflows. Add an assert that j > 0 before decrementing, since
j starts at i + 1 >= 1 and is never decremented below its
initial value.
CID#1548015
Follow-up for 8dcc66cefc
This is a followup to https://github.com/systemd/systemd/pull/41400 with
the final set of tweaks so that the new coccinelle `check-pinters-deref`
checker runs without failures.
It also includes a commit with some fixes for redundant asserts and the
final commit to make the assert() in the inner loop of qsort
POINTER_MAY_BE_NULL which will become a no-op - that is probably the
most controversial one, I hope the commit message explains the
trade-offs, I'm happy to drop it, I have no strong opinion either way.
Please see the individual commits/commit messages for more details, most
is (hopefully) relatively boring/mechanical.
When running in a console mode (interactive, native, or read-only),
propagate the host's $TERM into the VM by adding TERM= and
systemd.tty.term.hvc0= to the kernel command line.
TERM= is picked up by PID 1 and inherited by services on /dev/console
(such as emergency.service). systemd.tty.term.hvc0= is used by services
directly attached to /dev/hvc0 (such as serial-getty@hvc0.service) which
look up $TERM via the systemd.tty.term.<tty> kernel command line
parameter.
While systemd can auto-detect the terminal type via DCS XTGETTCAP, not
all terminal emulators implement this, so explicitly propagating $TERM
provides a more reliable experience. We skip propagation when $TERM is
unset or set to "unknown" (as is the case in GitHub Actions and some
other CI environments).
Previously this was handled by mkosi synthesizing the corresponding
kernel command line parameters externally.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
The network generator currently supports many of the options described
by dracut.cmdline(7), but not everything.
This commit adds support for the BOOTIF= option (and the related
rd.bootif= option) used in PXE setups.
This is implemented by treating BOOTIF as a special name/placeholder
when used as an interface name, and expecting a MAC address to be set in
the BOOTIF= parameter. The resulting .network file then uses MACAddress=
in the [Match] section, instead of Name=.
terminal_prepare_query() is called from terminal_get_size() which
operates on an explicitly passed fd — typically /dev/console opened
directly by PID 1 via reset_dev_console_fd(), or a service's TTY via
exec_context_apply_tty_size(). Using terminal_is_dumb() here is wrong
because it additionally checks on_tty(), which tests whether stderr is
a tty. PID 1's stderr may not be a tty (e.g. connected to kmsg or the
journal), causing terminal_is_dumb() to return true and skip the ANSI
query even though the fd we're operating on is a perfectly functional
terminal.
Use getenv_terminal_is_dumb() instead, which only checks $TERM, matching
what terminal_reset_ansi_seq() already does.
Also use it in terminal_get_cursor_position(), which also receives fds
to operate on.
terminal_get_size() writes ANSI escape sequences (CSI 18 and DSR
queries) to the output fd to determine terminal dimensions. This is
called during early boot via reset_dev_console_fd() and from service
execution contexts via exec_context_apply_tty_size().
Previously, these writes used loop_write() on a blocking fd, which
could block indefinitely if the terminal is not consuming data — for
example on a serial console with flow control asserted, or a
disconnected terminal. This is the same problem that was solved for
terminal_reset_ansi_seq() in systemd/systemd#32369 by temporarily
setting the fd to non-blocking mode with a write timeout.
Apply the same pattern here: set the output fd to non-blocking in
terminal_get_size() before issuing the queries, and restore blocking
mode afterward. Change the loop_write() calls in
terminal_query_size_by_dsr() and terminal_query_size_by_csi18() to
loop_write_full() with a 100ms timeout so writes fail gracefully
instead of hanging.
Also introduce the CONSOLE_ANSI_SEQUENCE_TIMEOUT_USEC constant for all
timeouts used across all ANSI sequence writes and reads (vt_disallocate(),
terminal_reset_ansi_seq(), and the two size query functions). 333ms
is now used for all timeouts in terminal-util.c.
Also introduce a cleanup function for resetting a fd back to blocking
mode after it was made non-blocking.
Switch runtime virtiofs mount configuration from systemd.mount-extra=
kernel command line parameters to the fstab.extra credential. This
avoids consuming kernel command line space (which is limited) and
matches the approach used by mkosi.
Each mount is added as an fstab entry in the format:
{tag} {destination} virtiofs {ro|rw},x-initrd.mount
If the user already specified a fstab.extra credential via
--set-credential= or --load-credential=, the virtiofs mount entries
are appended to it rather than conflicting.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
hmac_sha256() leaves four stack buffers containing key-derived material
(inner_padding, outer_padding, replacement_key, hash state) on the stack
after returning. The inner_padding and outer_padding arrays contain
key XOR 0x36 and key XOR 0x5c respectively, which are trivially
reversible to recover the original HMAC key.
This function is called with security-sensitive keys including the LUKS
volume key (cryptsetup-util.c), TPM2 PIN (tpm2-util.c), and boot secret
(tpm2-swtpm.c). The key material persists on the stack until overwritten
by later unrelated function calls.
Add CLEANUP_ERASE() to all four local buffers, following the same
pattern applied to tpm2-util.c in commit 6c80ce6 (PR #41394).
Some functions are in the hot path of a sort. While it is more
correct to also use assert() here for the passed pointers it is
also a runtime cost that should never be needed when e.g. qsort()
is used as it will always pass non-NULL pointers.
So for these critical functions use POINTER_MAY_BE_NULL() which
is compiled out and add a comment to them. Not all _compare()
functions are changed, it was a bit of a judgement call because
some (like ether_addr_compare) are not called with that much data.
But happy to get advise here and tweak accordingly.
Add the needed assert changes to make the code clean
for the new check-pointer-deref script.
Note that for `_cmp()` functions its a bit of a question
if we want `assert()` or `POINTER_MAY_BE_NULL()` here. The
former is more correct but it adds a (small) runtime hit. If
we use POINTER_MAY_BE_NULL() it is compiled out but strictly
speaking it is wrong (so at least we would need a comment).
Since v257, terminal_fix_size() is called during early boot via
console_setup() → reset_dev_console_fd() to query terminal dimensions
via ANSI escape sequences. This has caused intermittent boot hangs
where the system gets stuck with a blinking cursor and requires a
keypress to continue (see systemd/systemd#35499).
The function tries CSI 18 first, then falls back to DSR if that fails.
Previously, each method independently opened a non-blocking fd, disabled
echo/icanon, ran its query, restored termios, and closed its fd. This
created two problems:
1. Echo window between CSI 18 and DSR fallback: After CSI 18 times out
and restores termios (re-enabling ECHO and ICANON), there is a brief
window before DSR disables them again. If the terminal's CSI 18
response arrives during this window, it is echoed back to the
terminal — where the terminal interprets \e[8;rows;cols t as a
"resize text area" command — and the response bytes land in the
canonical line buffer as stale input that can confuse the DSR
response parser.
2. Cursor left at bottom-right on DSR timeout: The DSR method worked by
sending two DSR queries — one to save the cursor position, then
moving the cursor to (32766,32766) and sending another to read the
clamped position. If neither response was received (timeout), the
cursor restore was skipped (conditional on saved_row > 0), leaving
the cursor at the bottom-right corner of the terminal. The
subsequent terminal_reset_ansi_seq() then moved it to the beginning
of the last line via \e[1G, making boot output appear at the bottom
of the screen — giving the appearance of a hang even when the system
was still booting.
This commit fixes both issues:
- terminal_fix_size() now opens the non-blocking fd and configures
termios once for both query methods, so echo stays disabled for the
entire CSI 18 → DSR fallback sequence with no gap. tcflush(TCIFLUSH)
is called before each query to drain any stale input from the tty
input queue.
- The DSR method now uses DECSC (\e7) / DECRC (\e8) to save and restore
the cursor position via hardware, instead of querying it with a
separate DSR round-trip. All four sequences (DECSC, CUP to
bottom-right, DSR query, DECRC) are sent in a single write, so the
terminal processes DECRC and restores the cursor regardless of whether
userspace ever reads the DSR response. This eliminates the
cursor-at-bottom-right artifact on timeout and simplifies the read
loop to only need a single DSR response instead of two.
- The repeated setup boilerplate (dumb check, verify_same, fd_reopen,
termios save/disable) is extracted into terminal_prepare_query(),
shared by terminal_get_size_by_csi18(), terminal_get_size_by_dsr(),
and terminal_fix_size().
Fixes: systemd/systemd#35499
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Add TERMIOS_NULL sentinel, TermiosResetContext, and CLEANUP_TERMIOS_RESET()
macro (modeled after CLEANUP_ARRAY()) to automatically restore terminal
settings when leaving scope, replacing manual goto+tcsetattr patterns.
Migrate ask_string_full(), terminal_get_cursor_position(),
get_default_background_color(), terminal_get_terminfo_by_dcs(),
terminal_get_size_by_dsr() and terminal_get_size_by_csi18() to use the new
cleanup macro, removing the goto-based cleanup labels and replacing them
with direct returns.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Coverity flags (uint64_t) sc * (uint64_t) ps as a potential
overflow. Add an assert to make the bounds explicit for static
analyzers.
CID#1548042
Follow-up for eefc66aa8f
Coverity flags ALIGN(sizeof(struct passwd/group)) + bufsize as
potential overflows in the getpw/getgr helpers. Add asserts to
make the bounds explicit for static analyzers.
CID#1548047
CID#1548049
CID#1548069
CID#1548070
Follow-up for 75673cd8ae
Coverity flags offsetof(DirectoryEntries, buffer) + DIRENT_SIZE_MAX * 8
as a potential overflow. All operands are compile-time constants, so add
an assert_cc() to prove this at build time.
CID#1548020
Follow-up for 6393b847f4
Coverity flags 2 * space as a potential overflow. The space value
is bounded by prior allocation success, but add an explicit assert
to document this for static analyzers.
CID#1548056
Follow-up for 5e592c66bd
Coverity flags the x->start + x->nr and y->start + y->nr additions
as potential overflows. These are safe because uid_range_add_internal()
validates start + nr <= UINT32_MAX before inserting entries. Add asserts
to document this invariant for static analyzers.
CID#1548015
Follow-up for 8530dc4467
Coverity thinks _mntidb can be used uninitialized, but this
is not the case when r == 0. Add a bool variable to make it
clearer instead of reusing 'r' later, and an assert to guide
static analyzers.
CID#1644850
Follow-up for 5817c73391
It's a very common case (vfat...), and it's just too much noise. After
all the whole function exists primarily to deal with O_TMPFILE not being
availeble everywhere...
Inspired by the discussion in #41161.
Also change the order of flags to be more logical. First the option
to specify at what fields we look, then the option to specify how we
return their name, the the value, and finally what to do if the value
is missing.
Lennart Poettering
Zbigniew Jędrzejewski-Szmek
Daan De Meyer
Frantisek Sumsal
Michael Vogt
Luca Boccassi
Yaping Li
noxiouz
Nandakumar Raghavan
Ivan Shapovalov
Michal Rybecky
Nick Rosbrook