Commit Graph
70894 Commits
Author SHA1 Message Date
Lennart Poettering e0abffe600 update TODO 2024-02-12 12:04:18 +01:00
Lennart Poettering d02018afdb test: add brief test for prclock varlink interfaces and varlinkctl --collect 2024-02-12 12:04:18 +01:00
Lennart Poettering 24835e9933 varlinkctl: if "call" verb is used, imply "-j"
For the other verbs turning off JSON mode makes sense, but for "call"
not so much, after all the contents of a method call reply is JSON we
couldn't really show any other way.

Hence, when JSON output was not configured otherwise in "call", default
to the same as -j.
2024-02-12 12:04:18 +01:00
Lennart Poettering 7781d28fd7 varlinkctl: add new --collect switch
It exposes the varlink_collect() call we internally provide: it collects
all responses of a method call that is issued with the "more" method
call flag. It then returns the result as a single JSON array.
2024-02-12 12:04:18 +01:00
Lennart Poettering 9bca989183 varlink: rework varlink_collect()
This reworks varlink_collect() so that it is not just a wrapper around
varlink_observe(), varlink_bind_reply() and others. It becomes a first
class operation.

This has various benefits:

1. Memory management is normalized: the reply json variant is now
   tracked as part of the varlink object, and thus we do not pass
   ownership to the caller. This is just like we do it for simple method
   calls and removes a lot of confusion.
2. The bind reply/user data pointer can be used for user stuff, we'll
   not silently override this.
3. We enforce an overall time-out operation on the whole thing, so that
   this synchronous operation does no longer block forever.
2024-02-12 12:04:18 +01:00
Lennart Poettering 72226a2f95 varlink: properly return reply flags to callers
We so far have a reply flags return parameter on varlink_call_full(),
but we return 0 always. Let's fix that, and return the actual flags we
see.
2024-02-12 12:04:18 +01:00
Lennart Poettering 0430a11eb4 varlink: enforce a maximum size limit on replies collected via varlink_collect()
We should not allow servers to blow up client's memory without bounds,
hence set a (high) limit on replies we'll collect before failing.
2024-02-12 12:04:18 +01:00
Lennart Poettering 15138e7980 pcrlock: add basic Varlink interface
This can be used to make or delete a PCR policy via Varlink. It can also
be used to query the current event log in CEL format.
2024-02-12 12:04:18 +01:00
Lennart Poettering 9fe15ce84d pcrlock: split out generation of CEL objects into helper func
This way, we can reuse it later to generate Varlink replies

No change in behaviour, just some trivial split out.
2024-02-12 12:04:18 +01:00
Lennart Poettering 8279484a41 pcrlock: use log_setup() 2024-02-12 12:02:27 +01:00
Yu Watanabe f43330eace Merge pull request #30209 from yuwata/sd-journal-reduce-fstat
sd-journal: potentially reduce number of fstat call
2024-02-12 19:38:24 +09:00
Eric Daigle 321a8c595e firstboot: validate keymap entry
As described in #30940, systemd-firstboot currently does not perform
any validation on keymap entry, allowing nonexistent keymaps to be
written to /etc/vconsole.conf. This commit adds validation checks
based on those already performed on locale entry, preventing invalid
keymaps from being set.

Closes #30940

m
2024-02-12 10:23:38 +01:00
Frantisek Sumsal 3588c510d3 Merge pull request #31149 from YHNdnzj/restart-force-oneshot
core/service: allow RestartForceExitStatus= for oneshot service
2024-02-12 10:20:09 +01:00
Yu Watanabe ac5c938006 Merge pull request #30138 from yuwata/udev-processing-flag
udev: introduce ID_PROCESSING flag
2024-02-12 11:38:57 +09:00
Yu Watanabe 78643f26a2 test: show error messages to stderr
Otherwise, 'meson test' hides the messages on failure.
2024-02-11 20:55:47 +01:00
Yu Watanabe 1651c914df repart: fix typo
Follow-up for 1a0541d44c.
2024-02-12 02:52:37 +09:00
Yu WatanabeandCosta Tsaousis 63d2c7554e logs-show: get timestamp and boot ID only when necessary
Previously, get_display_timestamp() is unconditionally called even if we
will show logs in e.g. json format.
This drops unnecessary call of get_display_timestamp().

This also makes journal fields in each entry parsed only once in
output_short(). Still output_verbose() twice though.

This should improve performance of dumping journals.

Replaces #29365.

Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
2024-02-12 02:42:36 +09:00
Yu WatanabeandCosta Tsaousis cb2b0326ee sd-journal: drop to use Hashmap to manage journal files per boot ID
As reported at https://github.com/systemd/systemd/pull/30209#issuecomment-1831344431,
using hashmap in frequently called function reduces performance.
Let's replace it with a single array and bsearch.

Replaces #29366.

Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
2024-02-12 02:42:36 +09:00
Yu WatanabeandCosta Tsaousis f7f062bf9f tree-wide: set SD_JOURNAL_ASSUME_IMMUTABLE where appropriate
Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
2024-02-12 02:42:36 +09:00
Yu WatanabeandCosta Tsaousis bd1af1d5fb sd-journal: introduce SD_JOURNAL_ASSUME_IMMUTABLE flag
The flag can be used when it is not necessary to follow journal file
update.

Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
2024-02-12 02:41:28 +09:00
Yu Watanabe 51b2bcf835 sd-journal: make journal_file_read_tail_timestamp() notify to the caller that some new journal entries added
Tiny optimization for journal_file_find_newest_for_boot_id().
2024-02-12 01:27:50 +09:00
Yu Watanabe 99e6f682d3 sd-journal: cache last entry offset and journal file state
When the offset of the last entry object (or last object for journal
files generated by an old journald) is not changed, the timestamps
should be updated by journal_file_read_tail_timestamp() are unchanged.
So, we can drop to call fstat() in the function.

As, the journal header is always mapped, so we can read the offset and
journal file state without calling fstat.
Still, when the last entry offset is changed, we may need to call fstat()
to read the entry object. But, hopefully the number of fstat() call
can be reduced.
2024-02-12 01:23:16 +09:00
Yu Watanabe a89fd4a046 Revert "test: temporarily disable test_sysctl"
This reverts commit 8ed7800d7b.
2024-02-12 01:14:55 +09:00
Yu Watanabe 13176bfedc network: do not enter initialized state when the interface is being processed by udevd
Fixes #30056.
2024-02-12 01:14:55 +09:00
Yu Watanabe f30a47c45e udev-util: introduce device_is_processing() helper function 2024-02-12 01:14:48 +09:00
Yu Watanabe 1e7c8fe494 udev-util: use device_get_property_bool() at one more place 2024-02-12 01:13:50 +09:00
Yu Watanabe f6bda694f9 udev: introduce ID_PROCESSING=1 boolean property
This indicates that the device is being processed by udevd.
2024-02-12 01:09:21 +09:00
Yu Watanabe 4ebf678f1f Merge pull request #30260 from yuwata/sd-journal-cleanups
sd-journal: further cleanups
2024-02-12 01:07:11 +09:00
Frantisek Sumsal 8ceac11069 Revert "packit: temporarily build systemd without BPF stuff"
The latest Rawhide compose (20240210.n.1) finished successfully and made
it to the mirrors, so let's drop the bpftool workaround.

This reverts commit 8a0ec8852c.
2024-02-11 16:45:03 +01:00
Mike Yuan 189a08e83d core/service: allow RestartForceExitStatus= for oneshot services
I think this was just overlooked in #13754, which removed
the restriction of Restart= on Type=oneshot services.
There's no reason to prevent RestartForceExitStatus=
now that Restart= has been allowed.

Closes #31148
2024-02-10 21:19:36 +08:00
Mike Yuan a5d9df35ac man/systemd.service: document that Restart=always/on-success are refused
by oneshot services
2024-02-10 21:19:35 +08:00
Mike Yuan d67c51e386 core/service: make error msg match with conditions
This was discussed in
https://github.com/systemd/systemd/pull/13754#discussion_r333395362.
I think we should actually list "success" Restart= settings instead.
There are more error statuses than success ones after all, and this
list hasn't really changed for quite some time.
2024-02-10 21:19:35 +08:00
Daan De Meyer 1a0541d44c repart: Add --generate-fstab= and --generate-crypttab= options
These can be used along with two new settings MountPoint= and
EncryptedVolume= to write fstab and crypttab entries to the given
paths respectively in the root directory that repart is operating on.

This is useful to cover scenarios that aren't covered by the
Discoverable Partitions Spec. For example when one wants to mount
/home as a separate btrfs subvolume. Because multiple btrfs subvolumes
can be mounted from the same partition, we allow specifying MountPoint=
multiple times to add multiple entries for the same partition.
2024-02-10 13:07:02 +01:00
Luca Boccassi 1a383a881b Merge pull request #31273 from mrc0mmand/test-55-arch-gcc
test: make the MemoryHigh= limit a bit more generous with sanitizers
2024-02-10 11:37:53 +00:00
Luca Boccassi 769ca9ab55 Merge pull request #31261 from bluca/repart_engine
repart: support engines and providers
2024-02-09 21:28:59 +00:00
Frantisek Sumsal 76aa0d5db1 test: clean up the code a bit 2024-02-09 20:45:47 +01:00
Frantisek Sumsal 974fe6131f test: make the MemoryHigh= limit a bit more generous with sanitizers
When we're running with sanitizers, sd-executor might pull in a
significant chunk of shared libraries on startup, that can cause a lot
of memory pressure and put us in the front when sd-oomd decides to go on
a killing spree. This is exacerbated further on Arch Linux when built
with gcc, as Arch ships unstripped gcc-libs so sd-executor pulls in over
30M of additional shared libs on startup:

~# lddtree build-san/systemd-executor
build-san/systemd-executor (interpreter => /lib64/ld-linux-x86-64.so.2)
    libasan.so.8 => /usr/lib/libasan.so.8
        libstdc++.so.6 => /usr/lib/libstdc++.so.6
        libm.so.6 => /usr/lib/libm.so.6
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1
    libsystemd-core-255.so => /root/systemd/build-san/src/core/libsystemd-core-255.so
        libaudit.so.1 => /usr/lib/libaudit.so.1
            libcap-ng.so.0 => /usr/lib/libcap-ng.so.0
...
    libseccomp.so.2 => /usr/lib/libseccomp.so.2
    libubsan.so.1 => /usr/lib/libubsan.so.1
    libc.so.6 => /usr/lib/libc.so.6

~# ls -Llh /usr/lib/libasan.so.8 /usr/lib/libstdc++.so.6 /usr/lib/libubsan.so.1
-rwxr-xr-x 1 root root 9.7M Feb  2 10:36 /usr/lib/libasan.so.8
-rwxr-xr-x 1 root root  21M Feb  2 10:36 /usr/lib/libstdc++.so.6
-rwxr-xr-x 1 root root 3.2M Feb  2 10:36 /usr/lib/libubsan.so.1

Sanitized libsystemd-core.so is also quite big:

~# ls -Llh /root/systemd/build-san/src/core/libsystemd-core-255.so /usr/lib/systemd/libsystemd-core-255.so
-rwxr-xr-x 1 root root  26M Feb  8 19:04 /root/systemd/build-san/src/core/libsystemd-core-255.so
-rwxr-xr-x 1 root root 5.9M Feb  7 12:03 /usr/lib/systemd/libsystemd-core-255.so
2024-02-09 20:45:39 +01:00
Daan De Meyer 23f9af646b Merge pull request #29960 from CodethinkLabs/vmspawn/mkosi-features
vmspawn: add features required by mkosi
2024-02-09 18:39:54 +01:00
Lennart Poettering e90ec3a847 Merge pull request #31269 from poettering/vconsole-enodev
vconsole/pid1: handle ENODEV on /dev/console somewhat graceful
2024-02-09 18:04:09 +01:00
Luca Boccassi 7716498548 Merge pull request #31268 from poettering/bpf-device-fixes
bpf-devices: various smaller fixes
2024-02-09 16:34:52 +00:00
Yu Watanabe 8c244a5094 systemctl: fix typo
Follow-up for 1baa0415ae.
2024-02-10 00:34:08 +09:00
Yu Watanabe 07ae001f81 test: add tests for journal_file_next_entry() 2024-02-10 00:30:04 +09:00
Yu Watanabe 6c4ea35962 sd-journal: do not read unnecessary object
In journal_file_next_entry(), if the passed offset matches an entry object,
then generic_array_bisect() returns the object, but the object we
requested is the next (or previous) object. Hence, we should not validate
the object returned by generic_array_bisect(), otherwise it may fail
when the journal is corrupted.

Note the validity of the entry object that should be returned by
journal_file_next_entry() will be checked in the following generic_array_get().
So, when journal_file_next_entry() succeeds, the returned object is
always validated.
2024-02-10 00:29:16 +09:00
dependabot[bot] 0279c0abf3 build(deps): bump systemd/mkosi
Bumps [systemd/mkosi](https://github.com/systemd/mkosi) from dbce89aabda438ba58080366631b2c242e365f21 to 070528fec478fc93af7ec057a5d2fd0045123c99.
- [Release notes](https://github.com/systemd/mkosi/releases)
- [Changelog](https://github.com/systemd/mkosi/blob/main/NEWS.md)
- [Commits](https://github.com/systemd/mkosi/compare/dbce89aabda438ba58080366631b2c242e365f21...070528fec478fc93af7ec057a5d2fd0045123c99)

---
updated-dependencies:
- dependency-name: systemd/mkosi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-09 16:28:12 +01:00
Yu Watanabe 1b4e322cdb sd-journal: always put verified object into the chain cache
Let's consider the case that
- the first array contains valid entries,
- all entries in the second array are corrupted.

Then, when we are going to upwards, and a call of generic_array_bisect()
matches the last entry of the first array, then the second array was
cached with last_index == UINT64_MAX, instead of the first array with
its last entry.
Hence, when generic_array_bisect() is called next time, the function call
of test() always fail. So, the cache entry is mostly meaningless.

Let's always store valid cache entry.
2024-02-10 00:26:17 +09:00
Yu Watanabe 821d55a8d7 sd-journal: drop duplicated tests in generic_array_bisect()
When left == right (== 0), generic_array_bisec_step() does not update
left and right. So the following test is exactly the same.
2024-02-10 00:26:17 +09:00
Luca Boccassi 0a8264080a repart: support OpenSSL engines/providers for signing
The provider API which is new requires providers, which are not
widely available and don't work very well yet, so also use a
fallback with the legacy engine API.
2024-02-09 15:11:25 +00:00
Luca Boccassi dba0afa14e openssl: add helper to load key from provider/engine
It's not the literal private key, but EVP_PKEY becomes a reference
to the engine/provider that OpenSSL knows how to use later
2024-02-09 14:36:10 +00:00
Lennart Poettering 3a51cf673f bpf-devices: if a device node is referenced which doesn't exist, downgrade log message
Currently in many of our test cases you'll see a warning about a tun
device not being around. Let's make that quiet, since if there's no such
device there's no point in adding it to a policy anyway, and it makes
useless noise go away.

We keep the warning as a warning if a device node is missing for other
errors than ENOENT.
2024-02-09 15:32:10 +01:00
Lennart Poettering 958b73bef1 bpf-devices: normalize the return handling of functions that put together policy
under some conditions we suppress generating BPF programs. Let's
systematically return 0 when we do this, and 1 if we did actually
soething, instead of second guessing this in the caller.

This is not only more correct, but allows us to suppress BPF programs in
more cases in later commits.
2024-02-09 15:32:10 +01:00