prefix_roota() is something we should stop using. It is bad for three
reasons:
1. As it names suggests it's supposed to be used when working relative
to some root directory, but given it doesn't follow symlinks (and
instead just stupidly joins paths) it is not a good choice for that.
2. More often than not it is currently used with inputs under control of
the user, and that is icky given it typically allocates memory on the
stack.
3. It's a redundant interface, where chase_symlinks() and path_join()
already exist as better, safer interfaces.
Hence, let's start moving things from prefix_roota() to path_join() for
the cases where that's appropriate.
Currently, if journald coredumps, the coredump is written to
/var/lib/systemd/coredump but the coredump metadata is not written
to the journal meaning we can't find out about the coredump's
existence via the journal. This means that coredumpctl can't be
used to work with journald coredumps, as well as any other tools
that rely on journald to know about coredumps.
To solve the issue, let's have systemd-coredump try to write
systemd-journald coredump metadata to the journal. We have to be
careful though, since if journald coredumps, there's no active
reader on the receive end of the journal socket, so we have to make
sure we don't deadlock trying to write to the socket. To avoid the
deadlock, we put the socket in nonblocking mode before trying to
write to it.
We now have a local implementation in string-util-fundamental.c, but
it's useful at a lot of other places, hence let's give it a more
expressive name and share it across the tree.
Follow-up for: 8d9156660d
In function 'sd_id128_equal',
inlined from 'journal_file_verify' at ../src/libsystemd/sd-journal/journal-verify.c:1047:29:
../src/systemd/sd-id128.h:119:43: error: 'entry_boot_id.qwords[0]' may be used uninitialized [-Werror=maybe-uninitialized]
119 | return a.qwords[0] == b.qwords[0] && a.qwords[1] == b.qwords[1];
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/libsystemd/sd-journal/journal-verify.c: In function 'journal_file_verify':
../src/libsystemd/sd-journal/journal-verify.c:823:20: note: 'entry_boot_id.qwords[0]' was declared here
823 | sd_id128_t entry_boot_id;
| ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
entry_boot_id is only used when entry_monotonic_set has been set, and that's
only done in one place where entry_boot_id is also initalized.
Fixes a bug introduced by 578cd1855b.
The function `data_object_in_hash_table()` calls
`journal_file_move_to_object()` with `OBJECT_DATA`. Hence,
previously obtained pointer to a data object may be now invalid.
Fixes#23794.
Fixes: CID#1469712
CID 1469712 (#1 of 1): Unused value (UNUSED_VALUE)
returned_value: Assigning value from sd_id128_from_string(word + 2, &boot_id) to r here,
but that stored value is overwritten before it can be used.
The lookup "works", but is not useful. It was introduced in
9c66f52813.
And printf will NULL args is invalid was introduced in
5d1ce25728 when support for fds was initally
added :(
Given we have two different types for the journal object flags and the
Compression enum, let's make the latter a regular non-sparse enum, and
thus remove some surprises. We have to convert anyway between the two,
and already do via COMPRESSION_FROM_OBJECT().
The compression helpers are used both in journal code and in coredump
code, and there's a good chance we'll use them later for other stuff.
Let's hence move them into src/basic/, to make them a proper internal
API we can use from everywhere where that's desirable. (pstore might be
a candidate, for example)
No real code changes, just some moving around, build system
rearrangements, and stripping of journal-def.h inclusion.
Suggested by Daniele Nicolodi:
https://github.com/systemd/systemd/pull/23160#discussion_r855853716
This is possible only if the macro is never used in #if, but only in C code.
This means that all places that use #if have to be refactored into C, but we
reduce the duplication a bit, and C is nicer to read than preprocessor
conditionals.
Follow-up for da13d2ca07. Instead of having
separate definitions of the bitmask flags, just define DEFAULT_COMPRESSION_FOO=0|1
directly.
(It *should* be possible to do this more simply, but the problem is that
anything that is used in #if cannot refer to C constants or enums. This is the
simplest I could come up with that preserves the property that we don't use #ifdef.)
The return value from compress_blob() is changed to propagate the error instead
of always returning -EOPNOTSUPP. The callers don't care about the specific error
value. compress_blob_*() are changed to return the compression method on success, so
that compress_blob() can be simplified. compress_stream_*() and compress_stream() are
changed in the same way for consistency, even though the callers do not currently use
this information (outside of tests).
It's not trivial to figure out that this function does not search
for an exact match for the given needle, but whichever object in
the given direction that's closest to the needle.
Let's add a documentation blurb to explain how the function works
for to help future developers looking at this function.
Follow-up for https://github.com/systemd/systemd/commit/cd3c6322dbc6370448bafc216ee4e19e32a79d9e
journal-def.h should be self-contained too, as it represents the journal object ABI.
Duplicate the enums, as they also need to be in config.h for it to be self-contained,
and enums are not available to the preprocessor. Use an assert to ensure they don't
diverge.
Compression and decompression are controlled by the same build flag,
so if one wants to use, say, LZ4 to compress, ZSTD has to be disabled,
which means one loses the ability to read zstd-compressed journals.
Add a default-compression meson option, that allows to select any of
the available compression algorithms as the default.
This can be problematic especially when there's no more free disk
space. Consider the following:.
When disk space becomes sparse, writting to the system journal can lead to
error. In this case journald attempts to make room by rotating the journals,
which consists in archiving online journals and opening new ones.
However opening new files is likely to fail too and in this case
journal_file_open() leaves half initialized file around but in online
state. Then the error is propagated and journald switches into volatile mode.
Next time a new message is received by journald, it tries to open the
persistent system journal file to switch automatically back to persistent
mode.
When opening the system journal, journal_file_open(), called by
managed_journal_file_open_reliably(), finds the persistent system journal left
previously and assumes that it was uncleanly closed and considers it as
corrupted. The error is reported to managed_journal_file_open_reliably(), which
backs the file up and attempts to create a new system file, which fails and
leaves a corrupted system file again.
Since this is done for each message received by journald, /var/log/message can
be filled with backup files pretty quickly.
To prevent this, the patch makes sure to delete the newly created file in case
of error.
We so far had some magic logic in place that files we open for write
with size zero are freshly created. That of course is a bogus
assumption, in particular as this code deals with corrupted file systems
which oftentimes contain zero size inodes from left-over runs.
Let's fix this properly, and actually let the kernel tell us whether it
create the file or not.
The data type off_t can be 64 on 32 bit systems if they have large
file support. Since mmap expects a size_t with 32 bits as second
argument truncation could occur. At worst these huge files could
lead to mmaps smaller than the previous check for small files.
This in turn shouldn't have a lot of impact because mmap allocates
at page size boundaries. This also made the PAGE_ALIGN call in
open_mmap unneeded. In fact it was neither in sync with other mmap
calls nor with its own munmap counterpart in error path.
If such large files are encountered, which is very unlikely in these
code paths, treat them with the same error as if they are too small.
Lennart Poettering
Frantisek Sumsal
Daan De Meyer
matoro
Zbigniew Jędrzejewski-Szmek
Yu Watanabe
Shreenidhi Shedi
Luca Boccassi
Franck Bui
Jan Janssen
Tobias Stoeckmann