This reverts commit 490aa05ca1.
As commented https://github.com/systemd/systemd/pull/38569#discussion_r2284978273,
the commit makes autofs check bypassed. Before the commit, when
CHASE_NO_AUTOFS is set, we did not shortcut chasing paths, and refused
any autofs mount points in the path. However, with the commit, the flag
was swapped but even when CHASE_AUTOFS is unset, the autofs check may be
skipped.
To fix the issue, rather than swapping the flag, we should introduce
another flag, say CHASE_TRIGGER_AUTOFS. This revert the commit, and in a
later commit, the new flag will be introduced.
Since https://github.com/systemd/systemd/commit/c5de7b14ae2e08d267d8d75bc88934ac6aa7dcd6
file searching implies a new mount api syscall by default,
to trigger automounts.
This is problematic in NSS plugins, as they are dlopen'ed inside
processes by glibc, for two reasons.
First of all, potentially searching on a networked filesystem
automount could lead to nasty surprises, such as the process
responsible for setting up the network filesystem trying to
search on that same filesystem.
More importantly, the new mount api syscall was never part of
the filesystem seccomp filter that we provide by default, and
given mounting/remounting/bind mounting is one of the possible
ways to bypass sandboxing it is very likely not allowed when
custom filters are used in sandboxed processes, if they don't
need to do these operations otherwise.
The filesystem seccomp mask we provide has been updated, however
this only takes effect on the next restart of a service. When
systemd is upgraded via a package upgrade, the new nss plugin is
installed and will be immediately dlopen'ed by glibc when needed,
without waiting for the process to restart, which means the existing
seccomp filter applies, causing the filter to trigger.
Given it's not really possible for any arbitrary program to
predict which NSS modules glibc will load, given programs do not
configure that and instead nsswitch is set up by the sysadmin,
it's impossible to handle at each process level. It's also not
possible to know when it will be triggered, given the plugin
is not linked in each binary tools like need-restart cannot
even pre-emptively restart services that may be affected.
This means in practice, upgrading from systemd << v258 to >= v258
requires a reboot to avoid either subtle or catastrophic system
failures.
By avoiding to trigger automounts in nss-systemd we can avoid
both issues.
userdb drop-ins are searched for in:
/etc/userdb/
/run/userdb/
/run/host/userdb/
/usr/local/lib/userdb/
/usr/lib/userdb/
none of which are supported as automounts anyway.
Note that this happens only when the userdbd service is not running,
as otherwise nss-systemd will go through the varlink IPC, rather than
doing the searches in-process.
So invert CHASE_NO_AUTOFS to CHASE_AUTOFS and set it in the places where
we do want to trigger automounts, like looking for the ESP.
Follow-up for https://github.com/systemd/systemd/commit/c5de7b14ae2e08d267d8d75bc88934ac6aa7dcd6
Fixes https://github.com/systemd/systemd/issues/38565
Previously we didn't provide any install_callback to
sd_bus_add_match_async() so in case AddMatch() method call timed out we
destroyed the bus connection. This seems overly aggressive and simply
updating the sd_bus_track object accordingly should be enough.
Follow-up for 37ce3fd2b7.
Fixes#32381
It is not easy to understand what happens to a journal file
even with debug logs enabled. Add more dbg messages around operations
started by users to make it possible to follow the flow of operations.
As discussed in https://github.com/systemd/systemd/issues/38399, "ordinary"
systems can have the field table with a large number of values, causing journal
rotation to occur early. For example, audit generates a log of fields:
$ journalctl --fields | rg -c '^_?AUDIT'
114
It seems that the "structured log" capabilities of the journal are being use
more than in the past. Looking at some journal files on my system, it seems
the field hash table field is quite high in many cases:
$ build/test-journal-dump /var/log/journal/*/* | rg 'table fill'
Data hash table fill: 15.1%
Field hash table fill: 69.1%
Data hash table fill: 4.9%
Field hash table fill: 32.4%
Data hash table fill: 10.2%
Field hash table fill: 34.2%
Data hash table fill: 9.9%
Field hash table fill: 37.2%
Data hash table fill: 26.8%
Field hash table fill: 21.9%
Data hash table fill: 35.6%
Field hash table fill: 22.8%
Data hash table fill: 25.5%
Field hash table fill: 54.1%
Data hash table fill: 3.4%
Field hash table fill: 43.8%
Data hash table fill: 75.0%
Field hash table fill: 70.3%
Data hash table fill: 75.0%
Field hash table fill: 63.1%
Data hash table fill: 75.0%
Field hash table fill: 74.2%
Data hash table fill: 35.6%
Field hash table fill: 43.2%
Data hash table fill: 35.5%
Field hash table fill: 75.4%
Data hash table fill: 75.0%
Field hash table fill: 59.8%
Data hash table fill: 75.0%
Field hash table fill: 56.5%
Data hash table fill: 16.9%
Field hash table fill: 76.3%
Data hash table fill: 18.1%
Field hash table fill: 76.9%
Data hash table fill: 75.0%
Field hash table fill: 42.0%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 32.1%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 22.5%
Data hash table fill: 9.6%
Field hash table fill: 53.8%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 35.6%
Field hash table fill: 75.1%
Data hash table fill: 33.6%
Field hash table fill: 50.2%
Data hash table fill: 75.0%
Field hash table fill: 26.7%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 29.1%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 31.8%
Data hash table fill: 75.0%
Field hash table fill: 18.9%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 20.1%
Data hash table fill: 75.0%
Field hash table fill: 29.1%
Data hash table fill: 75.0%
Field hash table fill: 30.9%
Data hash table fill: 75.0%
Field hash table fill: 28.5%
Data hash table fill: 75.0%
Field hash table fill: 28.5%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 25.2%
Data hash table fill: 75.0%
Field hash table fill: 39.3%
Data hash table fill: 50.2%
Field hash table fill: 75.1%
Data hash table fill: 75.0%
Field hash table fill: 61.9%
Data hash table fill: 75.0%
Field hash table fill: 56.5%
Data hash table fill: 75.0%
Field hash table fill: 58.6%
Data hash table fill: 48.9%
Field hash table fill: 79.6%
Data hash table fill: 75.0%
Field hash table fill: 71.5%
Data hash table fill: 75.0%
Field hash table fill: 60.1%
Data hash table fill: 31.4%
Field hash table fill: 75.7%
Data hash table fill: 27.0%
Field hash table fill: 69.4%
Data hash table fill: 28.9%
Field hash table fill: 76.6%
Data hash table fill: 60.2%
Field hash table fill: 79.9%
Data hash table fill: 8.8%
Field hash table fill: 78.7%
Data hash table fill: 5.8%
Field hash table fill: 61.3%
Data hash table fill: 75.0%
Field hash table fill: 64.0%
Data hash table fill: 61.4%
Field hash table fill: 63.4%
Data hash table fill: 29.7%
Field hash table fill: 61.9%
Data hash table fill: 18.9%
Field hash table fill: 30.9%
Data hash table fill: 1.4%
Field hash table fill: 22.2%
Data hash table fill: 0.4%
Field hash table fill: 13.5%
Data hash table fill: 2.6%
Field hash table fill: 37.5%
Data hash table fill: 1.3%
Field hash table fill: 23.4%
Data hash table fill: 0.6%
Field hash table fill: 15.3%
Data hash table fill: 18.7%
Field hash table fill: 33.9%
Data hash table fill: 7.4%
Field hash table fill: 37.5%
Data hash table fill: 20.2%
Field hash table fill: 44.1%
Data hash table fill: 1.3%
Field hash table fill: 33.0%
Data hash table fill: 75.0%
Field hash table fill: 19.2%
Data hash table fill: 42.2%
Field hash table fill: 23.4%
Data hash table fill: 1.6%
Field hash table fill: 87.1%
Data hash table fill: 0.1%
Field hash table fill: 98.8%
Data hash table fill: 0.2%
Field hash table fill: 128.8%
Data hash table fill: 15.4%
Field hash table fill: 31.2%
Data hash table fill: 7.4%
Field hash table fill: 22.5%
Data hash table fill: 10.5%
Field hash table fill: 38.7%
Data hash table fill: 2.8%
Field hash table fill: 18.0%
Data hash table fill: 1.5%
Field hash table fill: 15.9%
Data hash table fill: 0.0%
Field hash table fill: 7.5%
Data hash table fill: 0.1%
Field hash table fill: 12.0%
Data hash table fill: 0.2%
Field hash table fill: 10.8%
Data hash table fill: 0.2%
Field hash table fill: 15.6%
Data hash table fill: 0.1%
Field hash table fill: 11.7%
Data hash table fill: 0.1%
Field hash table fill: 12.0%
Data hash table fill: 0.0%
Field hash table fill: 6.6%
Data hash table fill: 1.4%
Field hash table fill: 18.0%
Data hash table fill: 0.7%
Field hash table fill: 16.8%
Data hash table fill: 1.1%
Field hash table fill: 18.0%
Data hash table fill: 0.2%
Field hash table fill: 10.8%
Data hash table fill: 0.1%
Field hash table fill: 10.8%
Data hash table fill: 0.4%
Field hash table fill: 11.1%
Since filling of the field hash table to 75% normally causes file rotation,
let's double the default to make rotation happen less often.
We'll use 11kB more for the hash table, which should be fine, considering
that journal files are usually at least 8 MB.
Closes https://github.com/systemd/systemd/issues/38399.
It's easier to think about the size in "objects", not bytes. Let's convert to
bytes at the last moment.
Also drop some of the pointless size suffixes. In general, it's the size of the
variable that matters, not the constant that is written to it.
No functional change.
We had errno_to_name() which works for "known" errnos, and returns NULL for
unknown ones, and then ERRNO_NAME which always returns an answer, possibly just
a number as a string, but requires a helper buffer.
It is possible for the kernel to add a new errno. We recently learned that some
architectures define custom errno names. Or for some function to unexpectedly
return a bogus errno value. In almost all cases it's better to print that value
rather than "n/a" or "(null)". So let's use ERRNO_NAME is most error handling
code. Noteably, our code wasn't very good in handling the potential NULL, so
in various places we could print "(null)". Since this is supposed to be used
most of the time, let's shorten the names to ERRNO_NAME/errno_name.
There are a few places where we don't want to use the fallback path, in
particular for D-Bus error names or when saving the error name. Let's rename
errno_to_name() to errno_name_no_fallback() to make the distinction clearer.
To avoid the following error:
```
test_sd_device_one(/sys/devices/virtual/block/dm-9)
src/libsystemd/sd-device/test-sd-device.c:104: Assertion failed: Expected "sd_device_new_from_syspath(&dev, syspath)" to succeed, but got error: -19/ENODEV
```
This also updates outdates comment about CentOS CI.
Even when we receive an inotify event, there is no relevant event source
exists. In that case, we need to drop the event from the buffer,
otherwise we cannot escape from the loop.
Fixes#38265.
Continuation of #37960.
The same concern as expalined in #37960 exists also in
missing_syscall.h. If we use enough new glibc, a function we want to use
may be already provided by glibc, but our baseline glibc may not. And it
is hard to detect in our daily development.
This moves all prototypes of syscalls to relevant headers, and missing
syscall functions are defined in relevant .c files of libc wrapper. This
way, we can use usual header as is, e.g. when we want to write code with
`move_mount()`, we can simply use sys/mount.h without checking if it is
supported by our baseline glibc.
Then, move syscall definitions to the wrapper, and prototypes are moved
to relevant headers.
This also adds checks for add_key() and request_key(), as one day
glibc may be going to add some of them separatedly.
The check for fspick in meson.build is dropped, as it is currently
unused in our code.
This also moves
- basic/missing_bpf.h -> include/override/linux/bpf.h,
- basic/missing_keyctl.h -> include/override/linux/keyctl.h.
- pass our system include directories to make generators use our libc
wrappers and latest kernel headers,
- include relevant headers in generated gperf file,
- use files() rather than find_program(), as the result of
find_program() cannot be passed to 'input' of custom_target(),
- move generate-bpf-delegate-configs.py to src/core/, as it is only used
by libcore.
These source files uses symbols provided by sys/stat.h, e.g. struct stat,
S_IFREG, S_IFBLK, and so on. Let's explicitly include sys/stat.h where
necessary.
Glibc's fcntl.h includes bits/stat.h, which provides these symbols, so
these symbols can be used without explicitly including sys/stat.h. But,
based on the discussion in #37922, we should explicitly include relevant
headers, and should not rely on the indirect inclusion.
Similar to 4f18ff2e29, but for sys/stat.h.
Currently, when running "systemctl preset-all --root=xxx" in mkosi
to enable/disable units for initrds, the system presets are used.
The problem with this approach is that the system presets are written
for the system, and that is not necessarily ideal for an initrd, but we
still want to use the same packages in the initrd that we install in the
system, so let's introduce a separate directory for initrd presets which
is used to pick up preset files from when we detect that we're configuring
an initrd (by looking for /etc/initrd-release).
We also introduce a systemd preset file for the initrd, which is based on
the system one, except with all the stuff unnecessary for the initrd removed.
Follow-up for dbef4dd4f2. Everything that that
commit says is true, but — at least for me — it wasn't obvious why the code is
correct and we can do fixed-size allocations like new(struct inotify_data, 1).
In sd_event_source.child, we have 5 bools. If we make them each take one byte,
the structure size increases. So let's do that for the three of them, and leave
the other two (less frequently used) squished into the last byte. This allows
more efficient code to be generated, without changing the size of the struct:
$ diff -u <(objdump -S build/src/shared/libsystemd-shared-258.so.0|awk '/^static void source_io_unregister/,/^\}/') \
<(objdump -S build/src/shared/libsystemd-shared-258.so|awk '/^static void source_io_unregister/,/^\}/')
s->io.registered = false;
- 34d46f: 48 8b 45 d8 mov -0x28(%rbp),%rax
- 34d473: 0f b6 90 a4 00 00 00 movzbl 0xa4(%rax),%edx
- 34d47a: 83 e2 fe and $0xfffffffe,%edx
- 34d47d: 88 90 a4 00 00 00 mov %dl,0xa4(%rax)
- 34d483: eb 04 jmp 34d489 <source_io_unregister+0x1ca>
+ 34bffe: 48 8b 45 d8 mov -0x28(%rbp),%rax
+ 34c002: c6 80 a4 00 00 00 00 movb $0x0,0xa4(%rax)
+ 34c009: eb 04 jmp 34c00f <source_io_unregister+0x1be>
return;
We're still discussing whether we want to remove this or not, but
regardless of whether we end up removing it or not, it's something
we add ourselves and as such should not live in our headers that
override glibc headers, so let's move it to fd-util.h.
We use symbols provided by unistd.h without including it. E.g.
open(), close(), read(), write(), access(), symlink(), unlink(), rmdir(),
fsync(), syncfs(), lseek(), ftruncate(), fchown(), dup2(), pipe2(),
getuid(), getgid(), gettid(), getppid(), pipe2(), execv(), _exit(),
environ, STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO, F_OK, and their
friends and variants, so on.
Currently, unistd.h is indirectly included mainly in the following two paths:
- through missing_syscall.h, which is planned to covert to .c file.
- through signal.h -> bits/sigstksz.h, which is new since glibc-2.34.
Note, signal.h is included by sd-eevent.h. So, many source files
indirectly include unistd.h if newer glibc is used.
Currently, our baseline on glibc is 2.31. We need to support glibc older
than 2.34, but unfortunately, we do not have any CI environments with
such old glibc. CIFuzz uses glibc-2.31, but it builds only fuzzers, and
many files are even not compiled.
[1] says:
> Since 0.60.0 the name argument is optional and defaults to the basename of
> the first output
We specify >= 0.62 as the supported version, so drop the duplicate name in all cases
where it is the same as outputs[0], i.e. almost all cases.
[1] https://mesonbuild.com/Reference-manual_functions.html#custom_target
Otherwise, the result may point to outside of the root directory.
This also sets CONF_FILES_REGULAR and CONF_FILES_FILTER_MASKED, as the
callers will call fopen() or friends for the result, thus the enumerated
results must be non-empty regular files.
Yu Watanabe
Luca Boccassi
Michal Sekletar
Zbigniew Jędrzejewski-Szmek
DaanDeMeyer
Ubuntu
Mike Yuan
Luca Boccassi