Vacuuming unlinks and deallocates archived journal files. A reader
may have already selected an entry from such a file, so final
validation can fail after candidate selection succeeded.
If the selected file is now unlinked, remove it and retry candidate
selection. Preserve errors from linked files, including the EIO
reported after SIGBUS recovery from a truncated mapping.
compare_boot_ids() (from 262299dccb) skips
the tail entry realtime timestamp ordering when the compared locations
are from two different machines (and hence have two different machine
IDs) - in that case we fall through to a realtime comparison between the
two locations. This works fine except for journals that were imported
via systemd-journal-remote - the machine ID in the journal header is set
to the receiving machine's ID instead of the original one; but the
original machine ID is kept in the _MACHINE_ID= field.
This, however, defeats the check in compare_boot_ids() so we treat the
remote journals as local ones and try to compare the boots based on
their tail entry realtime timestamps, which results in grouping the
entries by their boot ID instead of interleaving them by their realtime
timestamp.
Given that we (transitively) call journal_file_read_tail_timestamp()
from compare_boot_ids(), where we both (re)set the newest_machine_id
field to the machine ID from the journal header and have the tail entry
at hand, let's try to get the _MACHINE_ID= field from the tail entry and
set newest_machine_id to that if it's present and valid. This way
compare_boot_ids() will correctly determine that the entries are from
two different machines and compare_locations() eventually falls through
to the entry realtime check, so we end up with properly interleaved
entries instead of entries grouped by boot ID.
The performance impact should be pretty negligible in vast majority of
cases, as we'd do this only when:
- opening a new file
- comparing entries across different boots
- advancing to a next location
The first case is a one-time thing only per journal. The second and
third case might call this a bit more frequently, but we fall through to
the _MACHINE_ID= parsing only if the journal file got a new entry since
the last check. The most significant impact would be on journals that
are being actively written into, but even then iterating through the
handful of typical fields is negligible compared to what the rest of the
function does. Also, since 8f8ab4bf98 the
journal_file_read_tail_timestamp() calls are ratelimited when walking
through the journal, lessening the impact even further.
Resolves: #34169
Since 262299dccb compare_locations() falls
through to a realtime-based comparison in compare_boot_ids() if the
compared entries have different boot IDs (and seqnum IDs). This can,
however, cause us to pick up the wrong entry/file on systems with
unreliable/missing RTC, where early-boot entries of a boot might have
earlier realtime timestamp compared to the previous boot.
For example, here's a minimal reproducer consisting of two journal files
containing three entries from two separate boot IDs:
$ journalctl --file one.journal -o export
__CURSOR=s=ba37c62fcacf406aad088166c4963cd1;i=1;b=c220efc91dd440deb89d82c98ddb11a5;m=186a0;t=1e8480;x=39d4323a8e5a248d
__REALTIME_TIMESTAMP=2000000
__MONOTONIC_TIMESTAMP=100000
__SEQNUM=1
__SEQNUM_ID=ba37c62fcacf406aad088166c4963cd1
_BOOT_ID=c220efc91dd440deb89d82c98ddb11a5
NUMBER=1
$ journalctl --file two.journal -o export
__CURSOR=s=9d15197efcc54d8ca5d6e7cce28373e2;i=1;b=686dc35e78f64b6ba126018ab6e8bae9;m=186a0;t=f4240;x=57db1c7bd38b37ed
__REALTIME_TIMESTAMP=1000000
__MONOTONIC_TIMESTAMP=100000
__SEQNUM=1
__SEQNUM_ID=9d15197efcc54d8ca5d6e7cce28373e2
_BOOT_ID=686dc35e78f64b6ba126018ab6e8bae9
NUMBER=2
__CURSOR=s=9d15197efcc54d8ca5d6e7cce28373e2;i=2;b=686dc35e78f64b6ba126018ab6e8bae9;m=30d40;t=2dc6c0;x=3513732159948113
__REALTIME_TIMESTAMP=3000000
__MONOTONIC_TIMESTAMP=200000
__SEQNUM=2
__SEQNUM_ID=9d15197efcc54d8ca5d6e7cce28373e2
_BOOT_ID=686dc35e78f64b6ba126018ab6e8bae9
NUMBER=3
The first file (one.journal) contains a single entry from the first boot
(c220efc91dd440deb89d82c98ddb11a5) and a realtime timestamp of 2s. The
second file (two.journal) contains two entries from the second boot
(686dc35e78f64b6ba126018ab6e8bae9) - the first entry has a realtime
timestamp of 1s (i.e. _earlier_ than the tail timestamp of the first
boot), and the second entry has a realtime timestamp of 3s (i.e. after
a time correction).
This works fine for forward iteration, but seeking directly to the
second entry is when things go south:
First entry:
$ python3 <<EOF
c = "s=ba37c62fcacf406aad088166c4963cd1;i=1;b=c220efc91dd440deb89d82c98ddb11a5;m=186a0;t=1e8480;x=39d4323a8e5a248d"
from systemd import journal; j = journal.Reader(path="."); j.seek_cursor(c); j.get_next(); print(j.test_cursor(c))
EOF
True
Second entry:
$ python3 <<EOF
c = "s=9d15197efcc54d8ca5d6e7cce28373e2;i=1;b=686dc35e78f64b6ba126018ab6e8bae9;m=186a0;t=f4240;x=57db1c7bd38b37ed"
from systemd import journal; j = journal.Reader(path="."); j.seek_cursor(c); j.get_next(); print(j.test_cursor(c))
EOF
False
That's because:
- we call sd_journal_seek_cursor(...) and parse it (seqnum, seqnum_id,
boot_id, realtime, ...)
- we call sd_journal_next() -> real_journal_next() which iterates over
all open journal files to find all candidate entries and pick the
best one
- for each journal file we call next_beyond_location()
- for next_beyond_location(one.journal) we get:
- last_direction is _DIRECTION_INVALID, so we call find_location_with_matches()
- we fall all the way through to journal_file_move_to_entry_by_realtime()
- here we try to find the first entry with realtime >= 1s; this file
has only one entry with realtime = 2s which matches
- we got a match and new_file is unset, so we set it to one.journal
- for next_beyond_location(two.journal) we get:
- we fall through to find_location_with_matches()
- here we have a matching seqnum, so we call journal_file_move_to_entry_by_seqnum()
- we got another match but new_file is set, so we call compare_locations(two.journal, one.journal):
- we fall through to compare_boot_ids()
- we get the tail timestamps of both journal files and return
CMP(second boot, first boot) -> CMP(3s, 2s) = 1
- this bubbles up back to real_journal_next() where found = false, so
new_file remains one.journal (the wrong journal file)
- this is followed by set_location(one.journal), which makes the
following sd_journal_test_cursor() call fail, as the
boot_id/seqnum_id/... of the current position don't match with the
cursor
To fix this, let's track the journal file that holds the exact cursor
match, and override the chosen candidate if needed once we process all
open journals.
Resolves: #31516
The updated parsing_hacks.h file uncovered a bunch of extra
things that the check-pointer-deref coccinelle script flags.
This commit fixes them to make the tree check-pointer-deref clean.
For libsystemd's headers, the changes made depend on whether the
header is installed or not. For installed headers, the only change
made is that commonly included headers were moved to _sd-common.h.
For these headers, there should be no noticeable change in behavior
when including them. For non-installed headers, includes were replaced
with forward declarations where possible as usual.
Split out of #37344.
The struct itself is generic, and can be used for other ID.
Let's rename it to more generic one.
No functional change, just refactoring and preparation for later
commits.
Fixes a regression introduced by e44f06065b.
After the offending commit, if a boot ID suffixed with an offset is
specified to --boot=, the boot ID was ignored.
This fixes the issue.
To fix the issue, this merges journal_find_boot_by_id() and
journal_find_boot_by_offset().
Those functions take a pointer to a timestamp and return a timestamp pointer,
so the reader would be justified to think that those are just getters. Rename
them to avoid confusion.
This is preparation for #28891, which adds a bunch more helpers around
"struct iovec", at which point this really deserves its own .c/.h file.
The idea is that we sooner or later can consider "struct iovec" as an
entirely generic mechanism to reference some binary blob, and is the
go-to type for this purpose whenever we need one.
Currently, all the logic related to writing journal files lives in
journal-file.c which is part of libsystemd (sd-journal). Because it's
part of libsystemd, we can't depend on any code from src/shared.
To allow using code from src/shared when writing journal files, let's
gradually move the write related logic from journal-file.c to
journald-file.c in src/journal. This directory is not part of libsystemd
and as such can use code from src/shared.
We can safely remove any journal write related logic from libsystemd as
it's not used by any public APIs in libsystemd.
This commit introduces the new file along with the JournaldFile struct
which wraps an instance of JournalFile. The goal is to gradually move
more functions from journal-file.c and fields from JournalFile to
journald-file.c and JournaldFile respectively.
This commit also modifies all call sites that write journal files to
use JournaldFile instead of JournalFile. All sd-journal tests that
write journal files are moved to src/journal so they can make use of
journald-file.c.
Because the deferred closes logic is only used by journald, we move it
out of journal-file.c as well. In journal_file_open(), we would wait for
any remaining deferred closes for the file we're about to open to complete
before continuing if the file was not newly created. In journald_file_open(),
we call this logic unconditionally since it stands that if a file is newly
created, it can't have any outstanding deferred closes.
No changes in behavior are introduced aside from the earlier execution
of waiting for any deferred closes to complete when opening a new journal
file.
Chris Down
Michael Renner
Frantisek Sumsal
Yu Watanabe
Michael Vogt
Daan De Meyer
Nick Rosbrook
Lennart Poettering
Zbigniew Jędrzejewski-Szmek