When journald is rotating a file, we'd like to log the reason at
LOG_INFO or higher instead of LOG_DEBUG. For journalctl --header,
logging the reason at a level higher than LOG_DEBUG doesn't really
make sense. To accomodate both use cases, make the log level used
by journal_file_rotate_suggested() configurable.
Let's define two helpers strdupa_safe() + strndupa_safe() which do the
same as their non-safe counterparts, except that they abort if called
with allocations larger than ALLOCA_MAX.
This should ensure that all our alloca() based allocations are subject
to this limit.
afaics glibc offers three alloca() based APIs: alloca() itself,
strndupa() + strdupa(). With this we have now replacements for all of
them, that take the limit into account.
udev: use netlink more aggressively
I'm pasting the comment from https://github.com/systemd/systemd/pull/20744#issuecomment-934485287
which is quite informative. The code wasn't changed significantly since then:
atenart commented 6 days ago:
> I ran tests without (93caec7) and with this PR (06735f2) on Fedora, having a few udev rules
> using attributes eligible to be cached and creating 50 veth on 4 CPUs. Although the time spent
> running the test is variable between runs, I generally saw an improvement when using this PR, e.g:
>
> 249-910-g93caec7:
> real 0m3.691s
> user 0m0.022s
> sys 0m1.338s
>
> 249-920-g06735f2:
> real 0m2.950s
> user 0m0.005s
> sys 0m0.399s
>
> On a different system than the one used above, I even saw a 40% improvement; results depend
> on many parameters (distro, udev rules, concurrent daemons accessing sysfs, etc.).
>
> Because it's quite hard to measure the improvement here (as the kernel behave differently between
> the two test cases), I also ran tests using a modified kernel not hitting the trylock logic. There was
> an improvement with this PR as well. (Take this with a grain of salt though, as the kernel was
> modified not using patches approved upstream).
Using OpenSSL brings in an additional dependency for all users of
libsystemd.so even though it's just one API that makes use of it.
The khash implementation is awkward as it requires context switches and
computation inside the kernel, thus leaving the process.
Remove both from libsystemd.so, and use exclusively the internal hmac fallback.
While this is not optimized, the sd-id128 API is not used in
performance-critical contexts where hardware acceleration would make a
noticeable difference.
Unfortunately fgetxattr() and flistxattr() don't work via O_PATH fds.
Let's thus add fallbacks to go via /proc/self/fd/ in these cases.
Also, let's merge all the various flavours we have here into singular
implementations that can do everything we need:
1. malloc() loop handling
2. by fd, by path, or combination (i.e. a proper openat() like API)
3. work on O_PATH
These functions are used pretty much independently of locale, i.e. the
only info relevant is whether th locale is UTF-8 or not. Hence let's
give this its own pair of .c/.h files.
Also,
- drop unnecessary +1 from buffer size, as IF_NAMESIZE or IFNAMSIZ
includes the nul at the end.
- format_ifname() does not update buffer on failure,
- introduces format_ifname_alloc(), FORMAT_IFNAME(), and their friends.
According to the documentation, Setting the data threshold to zero disables the
data threshold alltogether. Let's make sure we actually implement this behaviour
in sd_journal_enumerate_fields() by only applying the data threshold if it exceeds
zero.
LLVM 13 introduced `-Wunused-but-set-variable` diagnostic flag, which
trips over some intentionally set-but-not-used variables or variables
attached to cleanup handlers with side effects (`_cleanup_umask_`,
`_cleanup_(notify_on_cleanup)`, `_cleanup_(restore_sigsetp)`, etc.):
```
../src/basic/process-util.c:1257:46: error: variable 'saved_ssp' set but not used [-Werror,-Wunused-but-set-variable]
_cleanup_(restore_sigsetp) sigset_t *saved_ssp = NULL;
^
1 error generated.
```
In sd_journal_enumerate_fields(), we check if we've already handled
a field by checking if we can find it in any of the already processed
journal files. We do this by calling
journal_file_find_field_object_with_hash(), which compares the size,
payload and hash of the given field against all fields in a journal file,
trying to find a match. However, since we now use per file hash functions,
hashes for the same fields will differ between different journal files,
meaning we'll never find an actual match.
To fix the issue(), let's use journal_file_find_field_object() when one
or more of the files we're comparing is using per file keyed hashes.
journal_file_find_field_object() only takes the field payload and size
as arguments and calculates the hash itself using the hash function from
the journal file we're searching in.
Idea is that all public APIs should take reference on objects that get
exposed to user-provided callbacks. We take the reference as a
protection from callbacks dropping it. We used to do this also here in
sd_event_loop(). However, in cleanup portion of f814c871e6 this was
accidentally dropped.
journal_file_verify() uses a set of tmpfs files to create lists
of object positions by type.
The existing code used a bare write() call for every object
position written, incurring a syscall per listed object.
This commit encapsulates the bare file descriptors in FILE *'s
and replaces the bare write with fwrite, buffering the writes so
there's less syscalls.
Cached `journalctl --verify` tests showed a ~8% faster runtime
with this change on a release build, verifying 1.3GiB of
production journals across 16 files.
journal_file_verify() doesn't actually verify field object hashes
against their contents, despite journald storing a hash with the
payload. This commit adds that verification.
Yu Watanabe
Daan De Meyer
Jan Janssen
Zbigniew Jędrzejewski-Szmek
Lennart Poettering
Luca Boccassi
Frantisek Sumsal
Michal Sekletar
Vito Caputo