None of these conditions are real issues, but they can simply happen
because we just swtched from /run to /var as backend for logging and
there are old files from different boots with different systemd versions
and so on.
Let's not make more noise than necessary: still log, but not consider it
a warning, but just some normal thing.
We are handling these issues safely after all: by rotating and starting
anew, i.e. there's no reason to be concerned.
Although this slightly more verbose it makes it much easier to reason
about. The code that produces the tests heavily benefits from this.
Test lists are also now sorted by test name.
We nowadays check for ordering anyway at time of writing entries, hence
we don't have to do that at moment of opening, too.
Benefit of dropping this check: we can safely archive files from the
future instead of marking them as broken.
The header of the journal file contains a boot ID field that is
currently updated whenever we open the journal file. This is not ideal:
pretty often we want to archive a journal file, and need to open it for
that. Archiving a foreign journal file should not mark it as ours, it
should just change the status flag in the file header.
The boot ID in the header is aleady rewritten whenever we write a
journal entry to the file anyway, hence all this patch effectively does
is slightly "delay" when the boot ID in the header is updated: instead
of immediately on open it is updated on the first entry that is written.
Net effect: archived journal files don't all look like they were written
to on a boot newer then they actually were
And more importantly: the "tail_entry_monotonic" field suddenly becomes
useful, since we know which boot it belongs to. Generally, monotonic
timestamps without boot ID information are useless, and this fixes it.
A new (compatible) header flag marks file where the boot_id can be
understood this way. This can be used by code that wants to make use of
the "tail_entry_monotonic" field to ensure it actually can do so safely.
This also renames the structure definition in journal-def accordingly,
to indicate we now follow the stricter semantics for it.
Previously, we skip the entries before arg_lines
unconditionally, which doesn't behave correctly
when used with --grep. After this commit, when
a pattern is specified, we don't skip the entries
early, but rely on the count of the lines shown
to tell us when to stop. To achieve that we would
have to search backwards instead.
Fixes#25147
In journalctl we don't run the static destructors defined via
the STATIC_DESTRUCTOR_REGISTER() macro, since it requires a corresponding
static_destruct() call. In most cases this is handled by
the DEFINE_(TEST_)?MAIN*() macros, but journalctl defines its own main
function, so let's handle that as well.
$ valgrind --suppressions=valgrind.supp --show-leak-kinds=all --leak-check=full build/journalctl --no-pager -u system.slice -n 10 >/dev/null
==2778093== Memcheck, a memory error detector
==2778093== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==2778093== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==2778093== Command: build/journalctl --no-pager -u system.slice -n 10
==2778093==
==2778093==
==2778093== HEAP SUMMARY:
==2778093== in use at exit: 8,221 bytes in 4 blocks
==2778093== total heap usage: 458 allocs, 454 frees, 255,182 bytes allocated
==2778093==
==2778093== 13 bytes in 1 blocks are still reachable in loss record 1 of 4
==2778093== at 0x484586F: malloc (vg_replace_malloc.c:381)
==2778093== by 0x4DA256D: strdup (strdup.c:42)
==2778093== by 0x4ADB747: strv_extend_with_size (strv.c:544)
==2778093== by 0x405386: strv_extend (strv.h:45)
==2778093== by 0x40816F: parse_argv (journalctl.c:933)
==2778093== by 0x40EAB5: main (journalctl.c:2111)
==2778093==
==2778093== 16 bytes in 1 blocks are still reachable in loss record 2 of 4
==2778093== at 0x484578A: malloc (vg_replace_malloc.c:380)
==2778093== by 0x484A70B: realloc (vg_replace_malloc.c:1437)
==2778093== by 0x4ADB2A3: strv_push_with_size (strv.c:423)
==2778093== by 0x4ADB620: strv_consume_with_size (strv.c:496)
==2778093== by 0x4ADB770: strv_extend_with_size (strv.c:548)
==2778093== by 0x405386: strv_extend (strv.h:45)
==2778093== by 0x40816F: parse_argv (journalctl.c:933)
==2778093== by 0x40EAB5: main (journalctl.c:2111)
==2778093==
==2778093== LEAK SUMMARY:
==2778093== definitely lost: 0 bytes in 0 blocks
==2778093== indirectly lost: 0 bytes in 0 blocks
==2778093== possibly lost: 0 bytes in 0 blocks
==2778093== still reachable: 29 bytes in 2 blocks
==2778093== suppressed: 8,192 bytes in 2 blocks
==2778093==
==2778093== For lists of detected and suppressed errors, rerun with: -s
==2778093== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
If a service defines Delegate=yes, its subcgroup won't inherit the
LogFilterPatterns= option, because the option is stored on the unit's
cgroup attributes, not on the subcgroup.
Fixed by using the unit's cgroup attributes instead.
Let's ensure that entry seqnums remain stable and monotonic across the
entire runtime of the system, even if local storage is turned off. Let's
do this by maintainer a counter file in /run/ which we mmap() and
wherein we maintain the counter from early-boot on till late shutdown.
This takes inspiration of the kernel-seqnum file we already maintain
like that that tracks which kmsg messages we already processed. In fact,
we reuse the same code for maintaining it.
This should allow the behaviour entry seqnums to be more predictable, in
particular when journal local storage is turned off. Previously, we'd
maintain the seqnum simply by always bumping it to the maximum of the
last written entry seqnum plus one, and the biggest seqnum so far
written to the journal file on disk. If we'd never write a file on disk,
or if no journal file was existing during the initrd→seqnum transition
we'd completely lose the current seqnum position during daemon restarts
(such as the one happening during the switch-root operation).
This also will cause a journal file rotation whenever we try to write to
a journal file with multiple sequence number IDs, so that we know that
from early boot trhough the entire runtime we'll have stable sequence
numbers that do not jump, and thus can be used to determine "lost"
messages.
This is a follow-up for 1d8d483f59 and
makes the strict ordering by realtime clock within each journal file
optional, not mandatory. It then enables it for all journal files
written by journald, but leaves it off on others (for example those
written by journald-remote).
This relaxes the logic behind writing journal files to the status quo
ante for all cases where the journal files are not generated, but are
merged/processed/propagated. Typically when processing journal records
from many files ordering by realtime clock and monotonic clock are
contradictory, and cannot be universally guaranteed as the records are
interleaved. By enforcing strict rules we would thus end up generating
myriads of separate journal files, each with just a few records in them.
Hence, let's losen restrictions again, but continue to enforce them in
journald, i.e. when we original create the journal files locally.
Note that generally there's nothing really wring with having journal
files with non-monotonically ordered entries by realtime clock. Looking
for records will not be deterministic anymore, but that's inherent to a
realtime clock that jumps up and down. So you won't get the "only"
answer, but still *a* answer that is correct if you seek for a realtime
clock.
This also adds similar logic on the monotonic clock, which is also only
enabled when generating journal files locally. This should be harder to
trigger (as journald will generate the messages, and should run with a
stable boot id and monotonic clock), but let's better be safe than
sorry, and refuse on the lower layer what makes no sense, even if it's
unlikely the higher layer will ever generate records that aren't ordered
by their monotonic clock.
If output fields are specified, let's store this in a Set right-away,
instead of converting between strv and Set again and again for each
line.
This is not only faster, but also simpler and shorter.
clang puts the whole function signature in __PRETTY_FUNCTION__, which is
a bit excessive for something that can already be figured out by using
the line number.
The name was misleading because the function actually archives offline
journals, ie it doesn't try to remove them to make some room.
No functional change.
Before this patch the only way to prevent journald from reading the audit
messages was to mask systemd-journald-audit.socket. However this had main
drawback that downstream couldn't ship the socket disabled by default (beside
the fact that masking units is not supposed to be the usual way to disable
them).
Fixes#15777
Ensure all realtime timestamps in a journal file are strictly
ordered on write as a defense-in-depth measure. All known callers of
journal_file_append_entry and journal_file_copy_entry, which call this
function, should be able to handle the error by rotating the journal.
This is especially helpful for systems with RTC local time enabled,
where all log entries from initramfs might be recorded as several hours
later than it actually is, which won't get caught by journald during log
flushing. In those cases, the resulting inconsistency can cause libsystemd
to loop infinitely through journal files as observed in
`abrt-dump-journal-oops`.
We never use the return value, and it's confusing and kinda pointless
what we return there.
Hence drop it.
Originally noticed by: tristone13th <tristone13th@outlook.com>
Alternative to: #25810
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
Use LogFilterPatterns from the unit's cgroup xattr in order to keep or
discard log messages before writing them to the journal.
When a log message is discarded, it won't be written to syslog, console...
either.
When a native, syslog, or standard output log message is received,
systemd-journald will process it if it matches against at least one
allowed pattern (if any) and none of the denied patterns (if any).
When executed on a systemd with an empty /etc/machine-id,
test-journal-interleaving fails in test_sequence_numbers_one() when
re-opening the existing "two.journal". This is because opening the
existing journal file with managed_journal_file_open() causes
journal_file_verify_header() to be called. This function tries to
compare the current machine-id to the machine-id in the journal file
header, but does not handle the case where the machine-id is empty or
non-existent.
Check if we have an initialized machine-id before executing this portion
of the test.
The journal has a mechanism to log to the runtime journal if it fails to
log to the system journal. This mechanism is not triggered when the file
system becomes read-only. We enable it here.
When appending an entry fails if shall_try_append_again returns true,
the journal is rotated. If the FS is read-only, rotation will fail and
s->system_journal will be set to NULL. After that, when find_journal
will try to open the journal since s->system_journal will be NULL, it
will open the runtime journal.
Lennart Poettering
Yu Watanabe
Jan Janssen
Mike Yuan
Winterhuman
Daan De Meyer
Frantisek Sumsal
Luca Boccassi
Quentin Deslandes
Franck Bui
msizanoen1
Zbigniew Jędrzejewski-Szmek
Nick Rosbrook
Space Meyer
Richard Phibel
jcg