The Synchronize() function is just too useful for clients, so that we
can make "systemd-run -v --user" actually useful. Hence let's make the
socket accessible without privs. Deny most method calls however, except
for the Synchronize() call.
Previously, if the Synchronize() varlink call is issued we'd wait for
journald to become idle before returning success. That is problematic
however: on a busy system journald might never become idle. Hence, let's
beef up the logic to ensure that we do not wait longer than necessary:
i.e. we make sure we process any data enqueued before the sync request
was submitted, but not more.
Implementing this isn't trivial unfortunately. To deal with this
reasonably, we need to determine somehow for incoming log messages
whether they are from before or after the point in time where the sync
requested was received.
For AF_UNIX/SOCK_DGRAM we can use SO_TIMESTAMP to directly compare
timestamps of incoming messages with the timestamp of the sync request
(unfortunately only CLOCK_REALTIME).
For AF_UNIX/SOCK_STREAM we can call SIOCINQ at the moment we initiate
the sync, and then continue processing incoming traffic, counting down
the bytes until the SIOCINQ returned bytes have been processed. All
further data must have been enqueued later hence.
With those two mechanisms in place we can relatively reliably
synchronize the journal.
This also adds a boolean argument "offline" to the Synchronize() call,
which controls whether to offline the journal after processing the
pending messages. it defaults to true, for compat with the status quo
ante. But for most cases the offlining is probably not necessary, and is
cheaper to do without, hence allow not to do it.
So far we schduled kmsg events at higher priority than native/syslog
ones. But that's quite problematic, since it means that kmsg events can
drown out native/syslog log events. And this actually shows up in some
CI tests.
Address that, and schedule all three sources at the same priority, so
that the earlier event always is processed first, regarding which
protocol is used.
Let's optionally issue a Varlink Synchronize() call in --follow mode
when asked to terminate. This is useful so that the tool can be called
and it is guaranteed it processed all messages generated before the
request to exit before it exits.
We want this in "systemd-run -v" in particular, so that we can be sure
we are not missing any log output from the invoked service before it
exits
Allow callers to synchronize on the point in time where the journal file
watches are fully established, in --follow mode.
Tools can invoke journalctl using this, knowing that any log message
happening after the READY=1 is definitely going to be processed by the
journalctl invocation.
We soon want to add for sock_diag(7) netlink sockets. Those reuse the
same message type codes for request and response but with different
message formats. Hence we need to look at NLM_F_REQUEST to determine
which message policy to apply. Hence it is essential to know the flags
parameters right away when creating a message, since we cannot do early
validation otherwise.
This only adds support for setting the flags value right at the moment
of creation of the message object, it does not otherwise add
sock_diag(7) support, that is added in a later message.
This also corrects the flag for synthetic NLMSG_ERROR messages which
should not have the NLM_F_REQUEST flag set (since they are responses,
not requests).
Let's extend pid1's varlink interface and add a Describe method to get
the global Manager object information as a JSON object
(io.systemd.Manager.Describe).
Because the new varlink interface should be available on both the user
managers and the system manager, we also make the necessary changes to
expose a varlink server on user managers.
This PR is first part of https://github.com/systemd/systemd/pull/33965
with minimal changes to address feedback.
Fixes: #22654
Add support for a sysfail boot entry. Sysfail boot entries can be used
for optional tweaking the automatic selection order in case a failure
state of the system in some form is detected (boot firmware failure
etc).
The EFI variable `LoaderEntrySysFail` contains the sysfail boot loader
entry to use. It can be set using bootctl:
```
$ bootctl set-sysfail sysfail.conf
```
The `LoaderEntrySysFail` EFI variable would be unset automatically
during next boot by `systemd-boot-clear-sysfail.service` if no system
failure occured, otherwise it would be kept as it is and a system
failure reason will be saved to `LoaderSysFailReason` EFI variable.
`sysfail_check()` expected to be extented to support possibleconditions
when we should boot sysfail("recovery") boot entry.
Also add support for using a sysfail boot entry in case of UEFI firmware
capsule update failure [1]. The status of a firmware update is obtained
from the EFI System Resource Table (ESRT), which provides an optional
mechanism for identifying device and system firmware resources for the
purposes of targeting firmware updates to those resources.
Current implementation uses the value of LastAttemptStatus field from
ESRT, which describes the result of the last firmware update attempt for
the firmware resource entry. The field is updated each time an
`UpdateCapsule()` is attempted for an ESRT entry and is preserved across
reboots (non-volatile).
This can be be used in setups with support for A/B OTA updates, where
the boot firmware and Linux/RootFS might be updated synchronously.
The check is activated by adding "sysfail-firmware-upd" to loader.conf
[1]
https://uefi.org/specs/UEFI/2.10/23_Firmware_Update_and_Reporting.html
Let's extend pid1's varlink interface and add a Describe method to
get the global Manager object information as a JSON object
(io.systemd.Manager.Describe).
Because the new varlink interface should be available on both the
user managers and the system manager, we also make the necessary
changes to expose a varlink server on user managers.
Follow-up for f44e7a8c11
This breaks the rule stated at the beginning of help_sudo_mode():
> NB: Let's not go overboard with short options: we try to keep a modicum of compatibility with
> sudo's short switches, hence please do not introduce new short switches unless they have a roughly
> equivalent purpose on sudo. Use long options for everything private to run0.
I've always been reluctant to invoke the current user's shell in another
user's context, hence was fully grounded in `sudo -i`. With this bit in
place `run0` will finally be feature-complete on my side ;-)
The kernel sometimes sends uevents in a random order, so previously the
received events were not sorted by their seqnum. We determine which
event is ready for processing by using the assumption that queued events
are sorted by their seqnum. Let's sort the received events before queue
them, to make events processed in a correct ordering.
Add support for using a sysfail boot entry in case of UEFI firmware
capsule update failure [1]. The status of a firmware update is obtained from
the EFI System Resource Table (ESRT), which provides an optional mechanism
for identifying device and system firmware resources for the purposes of
targeting firmware updates to those resources.
Current implementation uses the value of LastAttemptStatus field from
ESRT, which describes the result of the last firmware update attempt for
the firmware resource entry. The field is updated each time an
UpdateCapsule() is attempted for an ESRT entry and is preserved across
reboots (non-volatile).
This can be be used in setups with support for A/B OTA updates, where
the boot firmware and Linux/RootFS might be updated synchronously.
[1] https://uefi.org/specs/UEFI/2.10/23_Firmware_Update_and_Reporting.html
Signed-off-by: Igor Opaniuk <igor.opaniuk@foundries.io>
You can configure the sysfail boot entry using the bootctl command:
$ bootctl set-sysfail sysfail.conf
The value will be stored in the `LoaderEntrySysFail` EFI variable.
The `LoaderEntrySysFail` EFI variable would be unset automatically
during next boot by `systemd-boot-clear-sysfail.service` if no
system failure occured, otherwise it would be kept as it is and a system
failure reason will be saved to `LoaderSysFailReason` EFI variable.
Signed-off-by: Igor Opaniuk <igor.opaniuk@foundries.io>
Add support for a sysfail boot entry. Sysfail boot entries can be
used for optional tweaking the automatic selection order in case a
failure state of the system in some form is detected (boot firmware
failure etc).
The EFI variable `LoaderEntrySysFail` holds the boot loader entry to
be used in the event of a system failure. If a failure occurs, the reason
will be stored in the `LoaderSysFailReason` EFI variable.
sysfail_check() expected to be extented to support possible
conditions when we should boot sysfail("recovery") boot entry.
Signed-off-by: Igor Opaniuk <igor.opaniuk@foundries.io>
This mostly makes sure we do something reasonable when our tool is
called from a boot of an entry that was already marked as definitely
"bad" on a previous boot. Such an entry we can return into a "good"
state, but we cannot return it into an "indeterminate" state, because
the status quo ante is already known.
Fixes: #37350
While we don't build these by default, all the source files still
get added to the compile_commands.json file by meson, which can confuse
tools as they might end up analyzing the source files twice or analyzing
the wrong one.
To avoid this issue, only define the static library target if the
corresponding option is enabled.
meson by default adds the current source and build directory as include
directories. Because we structure our meson code by gathering a giant dict
of everything we want to do and then doing all the actual target generation
in the top level meson.build, this behavior does not make sense at all because
we end up adding the top level repository directory as an include directory
which is never what we want.
At the same time, let's also make sure the top level directory of the build
directory is not an include directory, by moving the version.h generation
into the src/version subdirectory and then adding the src/version subdirectory
of the build directory as an include directory instead of the top level
repository directory.
Making this change means that language servers such as clangd can't get
confused when they automatically insert an #include line and insert
"#include "src/basic/fs-util.h" instead of "#include "fs-util.h".
Currently, when we want to add unit tests for code that is compiled into
an executable, we either compile the code at least twice (once for the
executable, and once for each test that uses it) or we create a static
library which is then used by both the executable and all the tests.
Both of these options are not ideal, compiling source files more than
once slows down the build for no reason and creating the intermediate
static libraries takes a lot of boilerplate.
Instead, let's use the extract_objects() method that meson exposes on
build targets. This allows us to extract the objects corresponding to
specific source files and use them in other executables. Because we
define all executables upfront into a dictionary, we integrate this into
the dictionary approach by adding two new fields:
- 'extract' takes a list of files for which objects should be extracted.
The extracted objects are stored in a dict keyed by the executable name
from which they were extracted.
- 'objects' takes the name of an executable from which the extracted
objects should be added to the current executable.
One side effect of this approach is that we can't build test executables
anymore without building the main executable, so we stop building test
executables unless we're also building the main executable. This allows
us to switch to using subdir_done() in all of these subdirectories to skip
parsing them if the corresponding component is disabled.
These changes get me down from 2439 => 2403 ninja targets on a full rebuild
from scratch.
Currently, when fuzzers are enabled, we run meson from within meson
to build the fuzzer executables with sanitizers. The idea is that
we can build the fuzzers with different kinds of sanitizers
independently from the main build.
The issue with this setup is that we don't actually make use of it.
We only build the fuzzers with one set of sanitizers (address,undefined)
so we're adding a bunch of extra complexity without any benefit as we
can just setup the top level meson build with these sanitizers and get
the same result.
The other issue with this setup is that we don't pass on all the options
passed to the top level meson build to the nested meson build. The only things
we pass on are extra compiler arguments and the value of the auto_features
option, but none of the individual feature options if overridden are passed on,
which can lead to very hard to debug issues as an option enabled in the top
level build is not enabled in the nested build.
Since we're not getting anything useful out of this setup, let's simplify
and get rid of the nested meson build. Instead, sanitizers should be enabled
for the top level meson.build. This currently didn't work as we were overriding
the sanitizers passed to the meson build with the fuzzer sanitizer, so we
fix that as well by making sure we combine the fuzzer sanitizer with the ones
passed in by the user.
We also drop support for looking up libFuzzer as a separate library as
it has been shipped builtin in clang since clang 6.0, so we can assume
that -fsanitize=fuzzer is available.
To make sure we still run the fuzzing tests, we enable the fuzz-tests option
by default now to make sure they still always run (without instrumentation unless
one of llvm-fuzz or oss-fuzz is enabled).
Lennart Poettering
Daan De Meyer
Ivan Kruglov
Eisuke KawashimaandEisuke Kawashima
André Monteiro
Mike Yuan
Yu Watanabe
Antonio Alvarez Feijoo
Igor Opaniuk