Yu Watanabe
8dea243077
Three independent fixes to code on `journalctl`'s per-entry read and output path. Each of them removes work that was never needed, rather than trading away anything. There is no behaviour change: the output is byte for byte identical for every format except `-o json`, where only the order of the keys within each object can shift, which is already nondeterministic between two runs of the unpatched binary (see the correctness section below). Nothing about the formats, tunables or public API changes. Measured on a 730 MB journal spread over 100 files (roughly 2M entries): | command | before | after | | |---|---:|---:|---:| | `journalctl -o cat` | 1.93s | 0.77s | **2.51x** | | `journalctl --grep=<no match>` | 1.97s | 0.82s | **2.40x** | | `journalctl -o export` | 4.06s | 1.73s | **2.35x** | | `journalctl -n 100000 -o short` | 0.86s | 0.43s | **2.00x** | | `journalctl -o json -n 200000` | 5.85s | 3.50s | 1.67x | | `journalctl -o short` (the default) | 3.32s | 1.96s | **1.69x** | | `journalctl -o verbose` | 5.99s | 3.61s | 1.66x | ### 1. `utf8_is_printable_newline()` decoded every character twice `utf8_encoded_valid_unichar()` decodes the character internally in order to validate it, and then throws the codepoint away. The caller immediately called `utf8_encoded_to_unichar()` again to get the very same codepoint back. So for every byte of every field printed, the UTF-8 decoder ran twice, only to answer whether the byte was a control character. This is fixed where it originates rather than at the call site: `utf8_encoded_valid_unichar_full()` optionally returns the character it had to decode anyway, so a caller that wants both the validity and the value decodes once. On top of that, eight bytes are tested at once for plain printable ASCII, which is what almost all journal content is. A word that does not qualify falls through to the per character path unchanged, so tabs, newlines and multibyte sequences behave exactly as before. On its own, against unmodified main: `-o export` 4.02s to 2.89s. ### 2. sd-journal read the clock once per journal file, per entry `sd_journal_next()` sweeps every open journal file to find the next entry, and `next_beyond_location()` rate limits its tail timestamp refresh per file. Since `ratelimit_below()` reads the clock itself, a journal spread over a hundred files meant a hundred `clock_gettime()` calls for every single entry iterated, nearly all of them only to conclude that the one second interval had not elapsed yet. This was the single largest cost in `journalctl`. In a profile of `-o export` it was over a fifth of the entire run, spent deciding not to do the thing it was guarding. Add `ratelimit_below_at()`, which takes the timestamp from the caller, and read the clock once per sweep instead. `ratelimit_below()` keeps its behaviour and becomes a wrapper. The timestamp is now sampled once per sweep rather than once per file. The interval being rate limited is one second and a sweep takes microseconds, so this makes no practical difference. On its own: `-o export` 4.02s to 2.93s, and `-o short` 3.30s to 2.14s. This is the largest single contributor for the default output mode. ### 3. The JSON output built a fresh, unsized hashmap for every entry `journal_entry_to_json()` creates an empty hashmap per entry and fills it with that entry's fields, of which there are typically around twenty. Because it starts empty it has to grow several times per entry, and every growth rehashes everything inserted so far. `-o json` spent about a quarter of its time in siphash, a third of that under `resize_buckets()`. Reserving room up front is a one line change. On its own: `-o json -n 200000` 5.71s to 4.64s.
System and Service Manager
Details
Most documentation is available on systemd's website.
Assorted, older, general information about systemd can be found in the systemd Wiki.
Information about build requirements is provided in the README file.
Consult our NEWS file for information about what's new in the most recent systemd versions.
Please see the Code Map for information about this repository's layout and content.
Please see the Hacking guide for information on how to hack on systemd and test your modifications.
Please see our Contribution Guidelines for more information about filing GitHub Issues and posting GitHub Pull Requests.
When preparing patches for systemd, please follow our Coding Style Guidelines.
If you are looking for support, please contact our mailing list, join our IRC channel #systemd on libera.chat or Matrix channel
Stable branches with backported patches are available in the stable repo.
We have a security bug bounty program sponsored by the Sovereign Tech Fund hosted on YesWeHack
Repositories with distribution packages built from git main are available on OBS, and also repositories with packages built from the latest stable release