When we're running with sanitizers, sd-executor might pull in a
significant chunk of shared libraries on startup, that can cause a lot
of memory pressure and put us in the front when sd-oomd decides to go on
a killing spree. This is exacerbated further on Arch Linux when built
with gcc, as Arch ships unstripped gcc-libs so sd-executor pulls in over
30M of additional shared libs on startup:
~# lddtree build-san/systemd-executor
build-san/systemd-executor (interpreter => /lib64/ld-linux-x86-64.so.2)
libasan.so.8 => /usr/lib/libasan.so.8
libstdc++.so.6 => /usr/lib/libstdc++.so.6
libm.so.6 => /usr/lib/libm.so.6
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1
libsystemd-core-255.so => /root/systemd/build-san/src/core/libsystemd-core-255.so
libaudit.so.1 => /usr/lib/libaudit.so.1
libcap-ng.so.0 => /usr/lib/libcap-ng.so.0
...
libseccomp.so.2 => /usr/lib/libseccomp.so.2
libubsan.so.1 => /usr/lib/libubsan.so.1
libc.so.6 => /usr/lib/libc.so.6
~# ls -Llh /usr/lib/libasan.so.8 /usr/lib/libstdc++.so.6 /usr/lib/libubsan.so.1
-rwxr-xr-x 1 root root 9.7M Feb 2 10:36 /usr/lib/libasan.so.8
-rwxr-xr-x 1 root root 21M Feb 2 10:36 /usr/lib/libstdc++.so.6
-rwxr-xr-x 1 root root 3.2M Feb 2 10:36 /usr/lib/libubsan.so.1
Sanitized libsystemd-core.so is also quite big:
~# ls -Llh /root/systemd/build-san/src/core/libsystemd-core-255.so /usr/lib/systemd/libsystemd-core-255.so
-rwxr-xr-x 1 root root 26M Feb 8 19:04 /root/systemd/build-san/src/core/libsystemd-core-255.so
-rwxr-xr-x 1 root root 5.9M Feb 7 12:03 /usr/lib/systemd/libsystemd-core-255.so
We treat all errors the same, hence let's just have a common handling
for that, and normalize how we propagate errors from shovel().
No change in behaviour, just some refactoring.
In interactive terminal mode, let's set a window title that reflects our
change of context to the target. Let's prefix it it with red/yellow
emoji dot in case we changed privileges.
Let's update the window title with an ANSI sequence if we can. We'll
insert a blue dot, to match the blue tinting of the terminal screen,
indicating that we are in a container.
This makes two changes:
1. it resets the background color not only on NL (aka LF) but also on
CR, but without erasing things to the end of the line. This increases
compatbility with tools such as "less" which use CR to jump back to
the beginning of the line.
2. previously we'd not process series of newlines or ansi sequences
without intermediate other characters correctly, we'd always assume
what follows is regular text. Fix that, and correctly determine the
right state from the subsequent character.
The terminate() method sends SIGHUP but this signal is not handled by
systemd-nspawn hence the process just exits leaving the container scope around
breaking futher test executions.
This patch sends SIGTERM instead which is a defined API to request
sytemd-nspawn to stop and release the container's resources properly.
Follow-up for 8a7032cfb1.
... i.e. apply nested config (exclusions and such) when executing R and D.
This fixes a long-standing RFE. The existing logic seems to have been an
accident of implementation. After all, if somebody specifies a config with
'R /foo; x /tmp/bar', then probably the goal is to remove stuff from under /foo,
but keep /tmp/bar. If they just wanted to nuke everything, then would not specify
the second item.
This also makes R and D use O_NOATIME, i.e. the access times of the directories
that are accessed will not be changed by the cleanup.
Obviously, we'll have to add this to NEWS and such.
Looking at the whole tmpfiles.d config in Fedora, this change has no effect.
The test cases are adjusted as appropriate. I also added another test case for
'R'/'D' with a file, just to test this code path more.
Replaces #20641.
Fixes#1633.
One of the three must always be specified, but they buried in a long list of
options in the output of --help. Make them more visible to draw the eye.
Also, drop "marked" from the description. It's supposed to mean "configured",
but it's a strange way to say that, and also it's generally obvious that the
program does what its configuration tells it to, and it's not going to remove
all files found on the system.
Previously, if given an absolute path, we would open the file, but when given a
relative path, we'd attempt to search the directories. If the user wants to open
a file from the search path, allowing paths is very confusing. E.g. with a path
like 'sysusers/foo.conf', we'd try to open '/etc/sysusers.d/sysusers/foo.conf',
'/run/sysusers.d/sysusers/foo.conf', …, and with '../foo.conf', we'd try to open
'/etc/sysusers.d/../foo.conf', '/run/sysusers.d/../foo.conf', …. This just isn't
useful, and in fact for a scheme like sysusers.d and tmpfiles.d where there we
have a flat directory with config files, only searching for plain names can
result in success. When a user specifies a relative path, it's more likely that
they wanted to open some local file. OTOH, to correctly open a local file, e.g.
one that they're just writing, this interface is also awkward, because something
like '$PWD/file.conf' has to be used to open a file with a relative path.
This patch changes the interface so that any path (i.e. an argument with "/") is
used to open a file directly, and only plain basenames are used for searching.
(Note that tpmfiles and sysusers are somewhat special here: their "config files"
make sense without the other config and users are likely to want to test them
without the other config. I was trying to do just that when writing a spec file
for a package and attempting to convert the existing scripts to sysusers and
tmpfiles. The same logic wouldn't apply for example to units or udev rules,
because they generally can only be interpreted with the whole rest of config
also available.)
I was annoyed that systemd-sysusers doesn't print any info when it opens a
config file. Its read_config_file() started out the same as the one in tmpfiles,
and then they diverged. The one in tmpfiles has that logging, hence the rework
to use it here too and get better logging. The two programs should provide
similar functionality, so using a common helper will make it easier to extend
them in tandem later.
No functional change apart from the log info.
The userdata argument (Context) is moved to the last position as requested in
the review.
a3451c2c4c added offline uid/gid support in a way
where the <root>/etc/passwd and <root>/etc/group would be read anew for each
configuration file that was parsed. The result would always be the same, so I
assume that this was an oversight. Let's use a global cache and and read the
file just once.
I was trying to run sysusers --replace, but the input file didn't have the right
suffix, and the message was very confusing. Let's split the message in two to
make it clearer that we care about the extension.
In journal_file_next_entry(), if the passed offset matches an entry object,
then generic_array_bisect() returns the object, but the object we
requested is the next (or previous) object. Hence, we should not validate
the object returned by generic_array_bisect(), otherwise it may fail
when the journal is corrupted.
Note the validity of the entry object that should be returned by
journal_file_next_entry() will be checked in the following generic_array_get().
So, when journal_file_next_entry() succeeds, the returned object is
always validated.
Let's consider the case that
- the first array contains valid entries,
- all entries in the second array are corrupted.
Then, when we are going to upwards, and a call of generic_array_bisect()
matches the last entry of the first array, then the second array was
cached with last_index == UINT64_MAX, instead of the first array with
its last entry.
Hence, when generic_array_bisect() is called next time, the function call
of test() always fail. So, the cache entry is mostly meaningless.
Let's always store valid cache entry.
The provider API which is new requires providers, which are not
widely available and don't work very well yet, so also use a
fallback with the legacy engine API.
Currently in many of our test cases you'll see a warning about a tun
device not being around. Let's make that quiet, since if there's no such
device there's no point in adding it to a policy anyway, and it makes
useless noise go away.
We keep the warning as a warning if a device node is missing for other
errors than ENOENT.
under some conditions we suppress generating BPF programs. Let's
systematically return 0 when we do this, and 1 if we did actually
soething, instead of second guessing this in the caller.
This is not only more correct, but allows us to suppress BPF programs in
more cases in later commits.
There's some unclarity whether major/minor of device nodes are supposed
to be "unsigned" or "dev_t". Various codebases assume the latter, but
glibc's major()/minor() types actually return a value typed to
"unsigned". On glibc dev_t is actually 64bit even if the kernel only
exposes 32bit. Hence this distinction kinda matters.
Let's clean things up a bit with handling: let's followe glibc's type
system here, and use unsigned (and not int).
Also let's pass invalid major/minor values around as UINT_MAX rather
than via pointers, to match how we usually do this, and to shorten our
code a bit. This is safe, since given the linux dev_t space being 32bit
only we can't possibly have a valid major or minor this hight, given
they must be smaller in size. While other archs disagree on the types of
major/minor, they also tend to have similar limits. In fact on FreeBSD
for example major()/minor() returns a signed int. Which would hence also
mean that UINT_MAX cannot be a valid major or minor.
Previously this function would log loudly in some cases but not in
others. Clean this up, and dont log at all, matching our coding style
which says we should either log in all error cases or in none.
Both callers of this function do logging already, hence no need to
duplicate it here.
Since 2e106312e2 the test unit fails with 'resources' result instead of
'exit-code', which the test didn't account for when running unprivileged.
Before 2e106312e2:
$ /root/systemd/build/test-path
Failed to start transient scope unit: Interactive authentication required.
Couldn't allocate a scope unit for this test, proceeding without.
...
-.slice: Failed to enable/disable controllers on cgroup /user.slice/user-1000.slice/session-1.scope, ignoring: Permission denied
app.slice: Failed to create cgroup /user.slice/user-1000.slice/session-1.scope/app.slice: Permission denied
-.slice: Failed to enable/disable controllers on cgroup /user.slice/user-1000.slice/session-1.scope, ignoring: Permission denied
app.slice: Failed to create cgroup /user.slice/user-1000.slice/session-1.scope/app.slice: Permission denied
...
line 151: path-exists.path: state = running; result = success (left: 29986250)
line 151: path-exists.service: state = start; result = success
path-exists.service: Main process exited, code=exited, status=219/CGROUP
path-exists.service: Failed with result 'exit-code'.
line 151: path-exists.path: state = running; result = success (left: 29985948)
line 151: path-exists.service: state = failed; result = exit-code
Failed to start service path-exists.service, aborting test: failed/exit-code
After 2e106312e2:
$ /root/systemd/build/test-path
Failed to start transient scope unit: Interactive authentication required.
Couldn't allocate a scope unit for this test, proceeding without.
...
-.slice: Failed to enable/disable controllers on cgroup /user.slice/user-1000.slice/session-1.scope, ignoring: Permission denied
app.slice: Failed to create cgroup /user.slice/user-1000.slice/session-1.scope/app.slice: Permission denied
-.slice: Failed to enable/disable controllers on cgroup /user.slice/user-1000.slice/session-1.scope, ignoring: Permission denied
app.slice: Failed to create cgroup /user.slice/user-1000.slice/session-1.scope/app.slice: Permission denied
path-exists.service: Failed to spawn executor: No such file or directory
path-exists.service: Failed to spawn 'start' task: No such file or directory
path-exists.service: Failed with result 'resources'.
Luca Boccassi
Frantisek Sumsal
Daan De Meyer
Lennart Poettering
Franck Bui
Zbigniew Jędrzejewski-Szmek
Yu Watanabe
dependabot[bot]
Sam Leonard