We keep adding fields to the header, and it's fine reading files with
different header sizes, as we check via the size if the fields we need
are included. However, let's be stricter when writing journal files than
when reading, and insist that the header structure in the file actually
matches our expectations. Refuse otherwise, so that a new file is
created after rotation that then matches our expectations.
This makes sure that mismatch in header size is treated exactly as
unknown "compatible" flags, which is our other mechanism to allow
extending the journal file format in a non-breaking way.
Let's pick up the boot ID early if unspecified, in
journal_file_append_entry(). This is symmetric to the fact that we
already pick up the monotonic timestamp in journal_file_append_entry()
if unspecified, and given that the monotonic clock is not too useful
without its boot ID it makes a lot of sense to pick them up at the same
time.
There are two relevant callers of journal_file_append_entry() right now:
journald (which leaves the boot ID unspecified) and journal-remote
(there are also some tests, but those don't matter too much). The former
calls it to store new entries in the journal file, the latter for
converting/processing/merging existing ones (where it passes along the
original boot ID). This new code hence only is relevant on the former,
and using the boot ID of the current system is the right choice for live
generated entries.
Note that this effectively changes little, since the lower-level
function journal_file_append_entry_internal() will copy boot ID stored
in the file header into all records if unspecified, and typically that's
the one of the local system. But strictly speaking this is not the right
thing to do, since we actually might end up appending to journal files
from previous boots. (The lower level function is indirectly used by
various tests, where the copying-from-header logic kinda makes sense
since they are detached from any live messages streaming in from the
host after all).
This is a follow-up for 1d8d483f59 and
makes the strict ordering by realtime clock within each journal file
optional, not mandatory. It then enables it for all journal files
written by journald, but leaves it off on others (for example those
written by journald-remote).
This relaxes the logic behind writing journal files to the status quo
ante for all cases where the journal files are not generated, but are
merged/processed/propagated. Typically when processing journal records
from many files ordering by realtime clock and monotonic clock are
contradictory, and cannot be universally guaranteed as the records are
interleaved. By enforcing strict rules we would thus end up generating
myriads of separate journal files, each with just a few records in them.
Hence, let's losen restrictions again, but continue to enforce them in
journald, i.e. when we original create the journal files locally.
Note that generally there's nothing really wring with having journal
files with non-monotonically ordered entries by realtime clock. Looking
for records will not be deterministic anymore, but that's inherent to a
realtime clock that jumps up and down. So you won't get the "only"
answer, but still *a* answer that is correct if you seek for a realtime
clock.
This also adds similar logic on the monotonic clock, which is also only
enabled when generating journal files locally. This should be harder to
trigger (as journald will generate the messages, and should run with a
stable boot id and monotonic clock), but let's better be safe than
sorry, and refuse on the lower layer what makes no sense, even if it's
unlikely the higher layer will ever generate records that aren't ordered
by their monotonic clock.
Ensure all realtime timestamps in a journal file are strictly
ordered on write as a defense-in-depth measure. All known callers of
journal_file_append_entry and journal_file_copy_entry, which call this
function, should be able to handle the error by rotating the journal.
This is especially helpful for systems with RTC local time enabled,
where all log entries from initramfs might be recorded as several hours
later than it actually is, which won't get caught by journald during log
flushing. In those cases, the resulting inconsistency can cause libsystemd
to loop infinitely through journal files as observed in
`abrt-dump-journal-oops`.
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
The compiler should recognize that these are constant expressions, but
let's better make this explicit, so that the linker can safely share the
initializations all over the place.
systemd-journald is prone to spamming logs if the system gets into
a messy state. Let's improve the situation by ratelimiting logs on
the hot code paths to 3 times per minute.
The function had just one caller and a name that didn't explain much.
Let's make it static and rename for clarity.
While at it, the only caller was not doing error handling correctly
— the function would potentially return a negative error value which
wasn't handled. In practice this couldn't happen, but let's remove
this ambiguity.
We have already made similar mistakes several times, e.g.
b8478c14c7, and
b596d06c385e104fc330288b791a56661f0c2d17. Let's document the function
invalidates previously read objects.
Previously, we'd iterate an entry array from start to end every time
we added an entry offset to it. To speed up this operation, we cache
the last entry array in the chain and how many items it contains.
This allows the addition of an entry to the chain to be done in
constant time instead of linear time as we don't have to iterate
the entire chain anymore every time we add an entry.
journal_file_data_payload() retrieves the payload of a Data object,
optionally decompressing it and checking to see if matches a given
field. This function replaces all the decompression code in the sd-journal
codebase with a single function.
This commit should not introduce any changes in sd-journal behavior.
To do this, we move EntryItem out of journal-def.h and turn it into
a host only struct in native endian mode so we can still use it to
ship the necessary info around.
Aside from that, the changes are pretty simple, we introduce some
extra functions to access the right field depending on the mode and
convert all the other code to use those functions instead of
accessing the raw fields.
We also drop the unused entry item hash field in compact mode. We
already stopped doing anything with this field a while ago, now we
actually drop it from the format in compact mode.
We also add an environment variable $SYSTEMD_JOURNAL_COMPACT that
can be used to disable compact mode if needed (similar to
$SYSTEMD_JOURNAL_KEYED_HASH).
This adds a new flag in preparation for incompatible journal changes
which will be gated behind this flag. The max file size of journal
files in compact mode is limited to 4 GiB.
Lennart Poettering
Franck Bui
msizanoen1
Zbigniew Jędrzejewski-Szmek
Yu Watanabe
Daan De Meyer
Luca Boccassi
David Tardon
Jan Janssen