Currently, when fuzzers are enabled, we run meson from within meson
to build the fuzzer executables with sanitizers. The idea is that
we can build the fuzzers with different kinds of sanitizers
independently from the main build.
The issue with this setup is that we don't actually make use of it.
We only build the fuzzers with one set of sanitizers (address,undefined)
so we're adding a bunch of extra complexity without any benefit as we
can just setup the top level meson build with these sanitizers and get
the same result.
The other issue with this setup is that we don't pass on all the options
passed to the top level meson build to the nested meson build. The only things
we pass on are extra compiler arguments and the value of the auto_features
option, but none of the individual feature options if overridden are passed on,
which can lead to very hard to debug issues as an option enabled in the top
level build is not enabled in the nested build.
Since we're not getting anything useful out of this setup, let's simplify
and get rid of the nested meson build. Instead, sanitizers should be enabled
for the top level meson.build. This currently didn't work as we were overriding
the sanitizers passed to the meson build with the fuzzer sanitizer, so we
fix that as well by making sure we combine the fuzzer sanitizer with the ones
passed in by the user.
We also drop support for looking up libFuzzer as a separate library as
it has been shipped builtin in clang since clang 6.0, so we can assume
that -fsanitize=fuzzer is available.
To make sure we still run the fuzzing tests, we enable the fuzz-tests option
by default now to make sure they still always run (without instrumentation unless
one of llvm-fuzz or oss-fuzz is enabled).
We add a default test setup that excludes the integration-tests suite
so that the integration tests don't run by default. This allows us to
get rid of $SYSTEMD_INTEGRATION_TESTS. Then, we add two extra setups:
'integration' and 'shell'. The 'integration' setup does not exclude the
integration-tests suite, and so can be used to run the integration tests.
The 'shell' setup does the same, but additionally sets $TEST_SHELL=1,
allowing to get rid of $TEST_SHELL in the docs.
Now that mkosi can automatically pick up its main configuration from
a mkosi/ subdirectory if it exists and there is no configuration in the
top level directory, let's make use of it to reduce the amount of clutter
in the top level directory of the repository.
This will also make it easier to install the mkosi configuration files as
part of the testing packages later on.
This allows us to use the regular settings instead of having to bother
with ToolsTreeXXX variants. It'll also allow us to share configuration
between the regular images and the tools tree image, which we'll make
use of in the next commit.
With meson 1.7.0, meson won't rebuild the world anymore when running
meson test so since meson 1.7.0 is in tumbleweed, debian testing, fedora
rawhide and Arch Linux, drop --no-rebuild from the docs since all the default
tools trees built by mkosi in the systemd repo will have meson 1.7.0 or newer
installed.
We keep --no-rebuild in CI because in CI we run with the same tools tree
distribution as the target distribution we're building and in those it's not
guaranteed for meson 1.7.0 to be available yet.
The advantage of dropping --no-rebuild is that meson will now reconfigure itself
if needed, which is required if we change or move around meson files, as otherwise
meson tends to explode spectacularly in hard to debug ways.
https://mesonbuild.com/Release-notes-for-1-7-0.html#test-targets-no-longer-built-by-default
This is a required first step before testing and it's not specifically
mentioned in the doc. This tripped me up for a while, so let's save the
trouble for the next person.
Currently, to run the integration tests, it's still necessary to
install various other build tools besides meson: A compiler, gperf,
libcap, ... which we want to avoid in CI systems where we receive
prebuilt systemd packages and only want to test them. Examples are
Debian's autopkgtest CI and Fedora CI. Let's make it possible for
these systems to run the integration tests without having to install
any other build dependency besides meson by extracting the logic
required to run the integration tests with meson into a separate
subdirectory and adding a standalone top-level meson.build file which
can be used to configure a meson tree with as its only purpose running
the integration tests.
Practically, we do the following:
- all the integration test directories and integration-test-wrapper.py
are moved from test/ to test/integration-test/.
- All the installation logic is kept out of test/integration-test/ or
any of its subdirectories and moved into test/meson.build instead.
- We add test/integration-test/standalone/meson.build to run the
integration tests standalone. This meson file includes
test/integration-test via a cute symlink hack to trick meson into
including a parent directory with subdir().
- Documentation is included on how to use the new standalone mode.
- TEST-64-UDEV-STORAGE and TEST-85-NETWORK are changed to generate separate
units for each testcase to make them behave more like the other integration
tests.
Yu Watanabe
Luca Boccassi
DaanDeMeyer
Allison Karlitskaya