The meson default for static_library() are:
build_by_default=true, install=false. We never interact with the
static libraries, and we only care about them as a stepping-stone towards
the installable executables or libraries. Thus let's only build them if
they are a dependency of something else we are building.
While at it, let's drop install:false, since this appears to be the default.
This change would have fixed the issue with lib_import_common failing
to build too: we wouldn't attempt to build it.
In practice this changes very little, because we generally only declare static
libraries where there's something in the default target that will make use of
them. But it seems to be a better pattern to set build_by_default to false.
The variable is not useful outside of the loop (it'll always be null
after the loop is finished), so we can declare it inline in the loop.
This saves one variable declaration and reduces the chances that somebody
tries to use the variable outside of the loop.
For consistency, 'de' is used everywhere for the var name.
Previously the MMapCache* was optionally NULL, which open would
handle by creating a new MMapCache* for the occasion.
This produced some slightly circuitous refcount-handling code in
the function, as well as arguably creating opportunities for
weirdness where an MMapCache* was intended to be supplied but
happened to be NULL, which this magic would then paper over.
In any case, this was basically only being utilized by tests,
apparently just to avoid having to create an MMapCache. So
update the relevant tests to supply an MMapCache and make
journal_file_open() treat a NULL MMapCache* as fatal w/assert.
This gets rid of the manual MMapCache ref/unref goop in
journal_file_{open,close}(), in favor of just letting the
JournalFile.cache_fd MMapFileDescriptor carry the baton.
Preparatory commit; before JournalFile can stop hanging onto its
copy of MMapCache, all these users need to find another way.
Most of the time these callers already have the MMapCache onhand,
so it's no big deal for them to just supply it.
journal_file_rotate() in particular needed to change, and it
seemed wise to not use the mmap_cache_fd_cache() accessor on
f->cache_fd, instead requiring the caller supply the cache to
use. This was done with an eye towards a potential future where
the journal_file_archive() isolates the cache_fd to a private
cache, which the newly rotated-to file wouldn't be allowed to
use. It's no biggie for the existing callers to just provide the
appropriate surviving cache.
Basically the mmap_cache_fd_cache() accessor was added just for
journal-verify.c's (ab)use of the mmap-cache. Which, if the
ugly singleton MMapCache assumption ever goes away, can be
cleaned up to simply use a separate MMapCache for those search
arrays.
Preparatory commit; callers manually ref/unref MMapCaches
alongside MMapFileDescriptor add/frees, when the latter should be
sufficient.
A subsequent commit will drop some of those manual MMapCache
reference hoop-jumping, leaving the lifecycle bound to
MMapFileDescriptors.
Disabling NOCOW when data has been written to a file doesn't work.
Instead, when we're done writing to a journal file (after archiving),
let's rewrite the file with COW enabled. This also takes care of
properly defragmenting the file.
With zstd compression level 3, journal files are compressed to 12%
of their original size with default journal settings.
As rewriting the file might take a while since we also do an fsync()
after the rewrite, this work is done in the offline thread to avoid
blocking the journald event loop.
When we archive a path, we rename the file to indicate this. However,
until now, we didn't actually update the path member of the corresponding
JournalFile instance. Let's make sure we also update this to avoid
misuse of the old path later on.
This change also requires we save the previous path in journal_file_rotate()
since we need to open a new file at the previous path.
With this change, the logic to write the final tag, emit the final
change notification and to offline the file moves from journal_file_close()
to journald_file_close(). Since all this logic is only executed when
the journal file is writable and all code that writes journal files
already uses journald_file_close() instead of journal_file_close(), this
change should not introduce any changes in behaviour.
Moving the offline related logic to journald-file.c allows us to use
code from src/shared in the offlining logic, more specifically, we can
use the file copying logic from copy.h to fix BTRFS filesystem compression
for journal files when archiving.
Currently, all the logic related to writing journal files lives in
journal-file.c which is part of libsystemd (sd-journal). Because it's
part of libsystemd, we can't depend on any code from src/shared.
To allow using code from src/shared when writing journal files, let's
gradually move the write related logic from journal-file.c to
journald-file.c in src/journal. This directory is not part of libsystemd
and as such can use code from src/shared.
We can safely remove any journal write related logic from libsystemd as
it's not used by any public APIs in libsystemd.
This commit introduces the new file along with the JournaldFile struct
which wraps an instance of JournalFile. The goal is to gradually move
more functions from journal-file.c and fields from JournalFile to
journald-file.c and JournaldFile respectively.
This commit also modifies all call sites that write journal files to
use JournaldFile instead of JournalFile. All sd-journal tests that
write journal files are moved to src/journal so they can make use of
journald-file.c.
Because the deferred closes logic is only used by journald, we move it
out of journal-file.c as well. In journal_file_open(), we would wait for
any remaining deferred closes for the file we're about to open to complete
before continuing if the file was not newly created. In journald_file_open(),
we call this logic unconditionally since it stands that if a file is newly
created, it can't have any outstanding deferred closes.
No changes in behavior are introduced aside from the earlier execution
of waiting for any deferred closes to complete when opening a new journal
file.
Entry arrays grow exponentially, so when archiving a journal file is
archived, it's very likely that the final entry array objects in each
entry array chain aren't fully used. Let's punch holes in the unused
parts so the filesystem can reclaim this unused space and use it for
something else.
Journal files have space allocated in 8MiB-aligned increments. This
can add up to substantial wasted space as many archived journals
accumulate without using all the allocated space. Let's truncate
journal files to their actually used size when archiving them to
reclaim this unused space.
As the mmap cache is not thread-safe, we can't call
journal_file_move_to_object() from the offline thread. Instead,
we use journal_file_read_object() which doesn't rely on the mmap
cache.
We would fail if the input file was corrupted:
build/test-journal-flush ./system@0005b7dac334f805-0021aca076ae5c5e.journal\~
journal_file_copy_entry failed: Bad message
Assertion 'r >= 0' failed at src/libsystemd/sd-journal/test-journal-flush.c:55, function main(). Aborting.
[1] 619472 IOT instruction (core dumped) build/test-journal-flush ./system@0005b7dac334f805-0021aca076ae5c5e.journal\~
Let's skip some "reasonable" errors.
Fixes#17963.
Code simplification; no idea why the Context structs were being
heap-allocated piecemeal. There aren't many and they're small,
and made even smaller with this commit.
This also gets rid of the Context.cache backreference by just
accompanying the Context* with MMapCache* to the few private
Context* operations, resolving the context ID to a Context* at
the single public entrypoint: mmap_cache_fd_get(), which asserts
on the value being within bounds. The Context.id member and
checks are also removed, as there's far less opportunity for
these things to become inconsistent now.
Of course this drops all context_{add,free}() related code,
making for a tasty diffstat.
If the iovec contains the same data more than once, we'll end up with
duplicate offsets in the items array. Let's make sure we remove any
duplicates before we store the items in an entry object.
The LIST_REMOVE() macro always assigns NULL to w->unused_prev,
meaning every time this window was in last_unused, the remainder
of the unused list was lost to the ether.
Turns out there's been a memory leak in journald after all, this
code has been there since at least 2013...
This actually causes mismatches between the header tail entry seqnum
and the last entry seqnum since when we revert the header seqnum, we
don't remove the entry object we added. If adding the entry object
itself fails, we don't need to revert the seqnum since it's never
incremented so let's remove this logic alltogether.
The argument names of methods under org.freedesktop.DBus.Properties and
org.freedesktop.DBus.Introspectable interfaces are specifies in D-Bus
specification[1]. They are:
org.freedesktop.DBus.Introspectable.Introspect (out STRING xml_data)
org.freedesktop.DBus.Properties.Get (in STRING interface_name,
in STRING property_name,
out VARIANT value);
org.freedesktop.DBus.Properties.Set (in STRING interface_name,
in STRING property_name,
in VARIANT value);
org.freedesktop.DBus.Properties.GetAll (in STRING interface_name,
out DICT<STRING,VARIANT> props);
sd-bus is using different argument names in the introspection document.
Usually this is not a problem but in case something tries to map the
argument names based on the introspection document to the position of
the arguments in the method call, then using names different than the
ones specified in the D-Bus specification is confusing.
So fix the names to match the D-Bus specification.
[1] https://dbus.freedesktop.org/doc/dbus-specification.html#standard-interfaces-introspectable
MMapFileDescriptor carries a reference to its originating
MMapCache, there's no value in supplying the
MMapFileDescriptor-centric functions a separate MMapCache.
A future commit will rename these functions to consistently use
an mmap_cache_fd_* prefix for improved clarity.
The existing verify_hash_table() function is renamed to
verify_data_hash_table() since it only verifies the data hash table.
The verify information is also made a little more detailed by splitting
one of the checks in two.
A little cleanup to make the next change easier. We're not moving to a
new Entry object in the for loop so there's no danger of changing the
Entry object window.
Yu Watanabe
Zbigniew Jędrzejewski-Szmek
Luca Boccassi
Vito Caputo
Daan De Meyer
Lennart Poettering
Miika Karanki