Because journal_file_next_entry_for_data() provides the first entry, while
journal_file_next_entry() actually provides the next entry of the input,
this also renames it to journal_file_move_to_entry_for_data().
Also, previously, on DIRECTION_UP the function did not fall back to the
'extra' entry when all entries linked in the chained array are broken.
This also fixes the issue, and now it fall back to the extra entry.
When we reach an empty array, there are at least two possibilities:
- journal file is corrupted,
- invalid index is requested.
We cannot distinguish them here, let's simply return earlier.
If there's corruption and we are going upwards, then the 'total'
must be decreased when we go to the previous array. However,
previously, we wrongly kept or increased the number. This fixes
the behavior.
This effectively reverts d9b61db922.
In the do-while loop, we do not read any other entry array object, hence
the current object is always in the mmap cache and not necessary to re-read it.
Consider the following situation:
- There are two journal files (x and y), that contains entries for two boots (X and Y).
- The journal file x contains entries of the boot X, and y contains
entries of Y.
- Nevertheless x does not contains entries of boot Y, it contains
the _BOOT_ID= data object of boot Y. Of course, the data object is not
referenced by any entries in the journal file x.
In such situation, when the current location of sd_journal is the head
of journal y, that is, the first entry of the boot Y,
sd_journal_previous() failed without this change, and
'journalctl --boot -NUM' for boot X failed.
Fixes#29275.
Otherwise, e.g. sd_journal_get_monotonic_usec() return an empty boot ID
when called for such a broken entry object.
Such a broken object may be stored when the system was not cleanly shutdown.
Fixes#29167.
Otherwise, sd_journal_previous() -> real_journal_next(DIRECTION_UP) ->
next_beyond_location() wrongly handles that previously we hit EOF of
the file, and returns 0 without finding a matching entry.
Fixes#29216.
The monotonic clock value consistency should only be checked if the boot
ID is the same as the last journal entry, and requiring that the current
boot ID be the same as the boot ID of the last entry is not necessary
for ensuring deterministic bisection as we already enforce strict
ordering of the real time clock value in a single journal file.
This fixes an issue where a new journal file is unnecessarily created
every boot, potentially wasting storage space.
With the old boot ID update behavior there's no way to compare the boot
ID of a new entry to be written with the boot ID of the last entry in
the journal in a performant manner, and therefore no way to determine if
monotonic clock value consistency checking is needed. Refuse to open
journal files without the new boot ID update behavior guarantee to
simplify consistency checking.
This is supposed to be a help for compilers to apply optimizations on
functions where they can't determine whether they are const/pure on
their own. For static, local functions the compiler can do this on its
own easily however, hence the decoration with pure/const is just noise.
Let's drop it, and let the compiler to its thing better.
(Use it for exported functions, since compilers can't 'reach-over' into
other modules to determine if they are pure, except if LTO is used)
After the commit 7a4ee86161,
sd_journal_next() following sd_journal_seek_tail() takes no-op,
and we need to call sd_journal_previous(). This may be useful in
some cases, e.g. to fix the issue explained in the previous commit.
If we fail to combine the new entry with a previous one, or update it in
the hashmap, we might later on attempt a double-free:
=================================================================
==10==ERROR: AddressSanitizer: attempting double-free on 0x611000039fc0 in thread T0:
SCARINESS: 42 (double-free)
#0 0x4a0962 in __interceptor_free /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3
#1 0x7f55e431d9f2 in _hashmap_clear /work/build/../../src/systemd/src/basic/hashmap.c:927:33
#2 0x7f55e431d4c8 in _hashmap_free /work/build/../../src/systemd/src/basic/hashmap.c:896:17
#3 0x4de1de in ordered_hashmap_free_free_free /work/build/../../src/systemd/src/basic/hashmap.h:120:24
#4 0x4de1de in ordered_hashmap_free_free_freep /work/build/../../src/systemd/src/basic/hashmap.h:434:1
#5 0x4de1de in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-catalog.c:26:1
#6 0x4de8b8 in NaloFuzzerTestOneInput (/build/fuzz-catalog+0x4de8b8)
#7 0x4fd8c3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
#8 0x4fd0aa in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:514:3
#9 0x4fe779 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:757:19
#10 0x4ff445 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:895:5
#11 0x4ee7af in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:912:6
#12 0x4ef078 in LLVMFuzzerRunDriver /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:925:10
#13 0x4deb35 in main (/build/fuzz-catalog+0x4deb35)
#14 0x7f55e3a32082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
#15 0x41f7cd in _start (/build/fuzz-catalog+0x41f7cd)
DEDUP_TOKEN: __interceptor_free--_hashmap_clear--_hashmap_free
0x611000039fc0 is located 0 bytes inside of 224-byte region [0x611000039fc0,0x61100003a0a0)
freed by thread T0 here:
#0 0x4a0962 in __interceptor_free /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3
#1 0x7f55e451493d in freep /work/build/../../src/systemd/src/basic/alloc-util.h:107:22
#2 0x7f55e451493d in finish_item /work/build/../../src/systemd/src/libsystemd/sd-journal/catalog.c:187:1
#3 0x7f55e4513e56 in catalog_import_file /work/build/../../src/systemd/src/libsystemd/sd-journal/catalog.c:313:45
#4 0x4de1be in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-catalog.c:23:16
#5 0x4de8b8 in NaloFuzzerTestOneInput (/build/fuzz-catalog+0x4de8b8)
#6 0x4fd8c3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
#7 0x4fd0aa in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:514:3
#8 0x4fe779 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:757:19
#9 0x4ff445 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:895:5
#10 0x4ee7af in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:912:6
#11 0x4ef078 in LLVMFuzzerRunDriver /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:925:10
#12 0x4deb35 in main (/build/fuzz-catalog+0x4deb35)
#13 0x7f55e3a32082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
DEDUP_TOKEN: __interceptor_free--freep--finish_item
previously allocated by thread T0 here:
#0 0x4a0c06 in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
#1 0x4de539 in malloc (/build/fuzz-catalog+0x4de539)
#2 0x7f55e42bf96b in memdup /work/build/../../src/systemd/src/basic/alloc-util.c:16:15
#3 0x7f55e451475d in finish_item /work/build/../../src/systemd/src/libsystemd/sd-journal/catalog.c:176:28
#4 0x7f55e4513e56 in catalog_import_file /work/build/../../src/systemd/src/libsystemd/sd-journal/catalog.c:313:45
#5 0x4de1be in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-catalog.c:23:16
#6 0x4de8b8 in NaloFuzzerTestOneInput (/build/fuzz-catalog+0x4de8b8)
#7 0x4fd8c3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
#8 0x4fd0aa in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:514:3
#9 0x4fe779 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:757:19
#10 0x4ff445 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:895:5
#11 0x4ee7af in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:912:6
#12 0x4ef078 in LLVMFuzzerRunDriver /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:925:10
#13 0x4deb35 in main (/build/fuzz-catalog+0x4deb35)
#14 0x7f55e3a32082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
DEDUP_TOKEN: __interceptor_malloc--malloc--memdup
SUMMARY: AddressSanitizer: double-free /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3 in __interceptor_free
Found by Nallocfuzz.
Recent gcc versions have started to trigger false positive
maybe-uninitialized warnings. Let's make sure we initialize
variables annotated with _cleanup_ to avoid these.
This reverts commit 1db6dbb1dc.
The original patch was reverted because of issue #25369. The issue was created
because it wrongly assumed that sd_journal_seek_tail() seeks to 'current' tail.
But in fact, only when a subsequent sd_journal_previous() is called that it's
pointing to the tail at that time. The concept of 'tail' in sd_journal_seek_tail()
only has a logical meaning, and a sd_journal_previous is needed. In fact, if we
look at the codes in journalctl, we can see sd_journal_seek_tail() is followed by
sd_journal_previous(). By contrary, a sd_journal_next() after a 'logical' tail does
not make much sense. So the original patch is correct, and projects that are
using 'sd_journal_next()' right after 'sd_journal_seek_tail()' should do fixes
as in https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2823#note_1637715.
Yu Watanabe
Luca Boccassi
Mike Yuan
msizanoen
Zbigniew Jędrzejewski-Szmek
Lennart Poettering
Frantisek Sumsal
Daan De Meyer
Chen Qi