This is not actually used so it doesn't really matter in
practice and the fields are used anyway, but for cleanliness
fix it
Reported on yeswehack.com as YWH-PGM9780-165
Follow-up for 5699a1689b
We have two places where those fields can be set: config and the dbus
interface. Let's clamp down on the number in both places. But in principle, we
could also be upgrading (through serialization/deserialization) from an older
systemd which didn't enforce this limit, so also check on deserialization. A
user could have a unit with lots and lots of ExtraFields, but not enough to
cause the issue in #40916. To handle this gracefully, ignore the extra fields,
like we do in the parser. Where the field is used, assert that we are within
the expected bounds.
Fixes#40916.
Reproducer:
$ python3 -c 'from pydbus import SystemBus; from gi.repository import GLib; SystemBus().get("org.freedesktop.systemd1", "/org/freedesktop/systemd1").StartTransientUnit("crash.service", "fail", [("ExecStart", GLib.Variant("a(sasb)", [("/bin/true", ["/bin/true"], False)])), ("LogExtraFields", GLib.Variant("aay", [b"F%d=x" % i for i in range(140000)]))], [])'
Traceback (most recent call last):
File "<string>", line 1, in <module>
from pydbus import SystemBus; from gi.repository import GLib; SystemBus().get("org.freedesktop.systemd1", "/org/freedesktop/systemd1").StartTransientUnit("crash.service", "fail", [("ExecStart", GLib.Variant("a(sasb)", [("/bin/true", ["/bin/true"], False)])), ("LogExtraFields", GLib.Variant("aay", [b"F%d=x" % i for i in range(140000)]))], [])
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/site-packages/pydbus/proxy_method.py", line 102, in __call__
raise error
File "/usr/lib/python3.14/site-packages/pydbus/proxy_method.py", line 97, in __call__
result = instance._bus.con.call_sync(*call_args)
gi.repository.GLib.GError: g-dbus-error-quark: GDBus.Error:org.freedesktop.DBus.Error.InvalidArgs: Too many extra log fields. (16)
Currently mount options are handled in such a way that the first
definition for a given partition wins, and documented as such.
Change them so that they behave like other options, and the
last specified wins.
Applies to RootImageOptions=, MountImages= and ExtensionImages=.
Switch from a linked list to an array indexed by the partition
specifier to store them.
Transparent Hugepages (THP) is a Linux kernel feature that manages
memory using larger pages (2MB on x86, compared to the default 4KB).
The main goal is to improve memory management efficiency and system
performance, especially for memory-intensive applications.
However, it can cause drawbacks in some scenarios, such as memory
regression and latency spikes. THP policy is governed for the entire
system via /sys/kernel/mm/transparent_hugepage/enabled.
However, it can be overridden for individual workloads via prctl(2)
call.
MemoryTHP= is used to disable THPs at exec-invoke to stop
providing THPs for workloads where the drawbacks outweigh the advantages.
When set to "disable", MemoryTHP= disables THPs completely for the
process, irrespecitive of global THP controls.
When set to "madvise", MemoryTHP= disables THPs for the process except
when specifically madvised by the process with MADV_HUGEPAGE or MADV_COLLAPSE.
* Do not interleave root_directory_as_fd with stdio fields
* Do not use different serialization key for different modes
pointing to same path
* Escape stdio file paths (as per
9be46b1da8)
Follow-up for 7c0afcdde2
Addresses https://github.com/systemd/systemd/pull/40060#issuecomment-3641288267
As commented, the unknown values should have been filtered out
in pid1's initial parsing already, and the communication between
pid1 and executor is entirely internal which makes the graceful
practice counterproductive.
Usually we gracefully ignore unknown configuration parameters, so that
service files can be written by upstreams and used across a variegated
range of distributions with various versions of systemd, to avoid
forcing users to the minimum common denominator and only adding settings
that are supported by the oldest distro supported.
Image policies do not behave like this, and any unknown partition or
policy designator causes the whole unit to fail to parse and a hard
error.
Change it so that parsing RootImagePolicy and friends via unit file or
D-Bus logs but otherwise ignores unknown specifiers, like other options
do.
This allows us to add new specifiers in the future, and users to adopt
them immediately.
Follow-up for d452335aa4
This allows a service to reuse the user namespace created for an
existing service, similarly to NetworkNamespacePath=. The configuration
is the initial user namespace (e.g. ID mapping) is preserved.
RootDirectory= but via a open_tree() file descriptor. This allows
setting up the execution environment for a service by the client in
a mount namespace and then starting a transient unit in that execution
environment using the new property.
We also add --root-directory= and --same-root-dir= to systemd-run to
have it run services within the given root directory. As systemd-run
might be invoked from a different mount namespace than what systemd is
running in, systemd-run opens the given path with open_tree() and then
sends it to systemd using the new RootDirectoryFileDescriptor= property.
The distinction between fdstore and extra fds is only meaningful
to struct Service. As far as executor is concerned they're just
some fds to pass to the service. Let's just merge it hence,
for the sake of simplicity.
Add a new option PrivateBPF= to mount a new instance of bpffs within a
namespace.
PrivateBPF= can be set to "no" to use the host bpffs in readonly mode
and "yes" to do a new mount.
The mount is done with the new fsopen()/fsmount() API because in future
we'll hook some commands between the two calls.
- use unsigned for the return value of read_nr_open(), as it does not
fail, and the kernel internally uses unsigned for the value,
- when bumping the value by PID1, let's start from the kernel's maximum
value defined in fs/file.c. The maximum value should be mostly an API
of the kernel, but may changed in a future, hence still try several
times if we fail to bump the value.
Co-authored-by: Jared Baur <jaredbaur@fastmail.com>
Co-authored-by: John Rinehart <johnrichardrinehart@gmail.com>
We use symbols provided by unistd.h without including it. E.g.
open(), close(), read(), write(), access(), symlink(), unlink(), rmdir(),
fsync(), syncfs(), lseek(), ftruncate(), fchown(), dup2(), pipe2(),
getuid(), getgid(), gettid(), getppid(), pipe2(), execv(), _exit(),
environ, STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO, F_OK, and their
friends and variants, so on.
Currently, unistd.h is indirectly included mainly in the following two paths:
- through missing_syscall.h, which is planned to covert to .c file.
- through signal.h -> bits/sigstksz.h, which is new since glibc-2.34.
Note, signal.h is included by sd-eevent.h. So, many source files
indirectly include unistd.h if newer glibc is used.
Currently, our baseline on glibc is 2.31. We need to support glibc older
than 2.34, but unfortunately, we do not have any CI environments with
such old glibc. CIFuzz uses glibc-2.31, but it builds only fuzzers, and
many files are even not compiled.
As it is defined as
```
static inline int parse_cpu_set(const char *rvalue, CPUSet *cpu_set){
return parse_cpu_set_full(rvalue, cpu_set, false, NULL, NULL, 0, NULL);
}
```
Our baseline is v5.4 and cgroup v2 is enforced now,
which means CPU accounting is cheap everywhere without
requiring any controller, hence just remove the directive.
If DefaultDependencies=no, /var/ may not be mounted yet when the service
is being started. Previously, In such case, if the service has
PrivateTmp=disconnected, the service manager created /var/tmp/ on the
root filesystem and mounted the disconnected private tmpfs there. That
poluted the root filesystem and disturbed gpt-auto-generator on next
boot, as /var/ would not be empty anymore. See issue #37258.
This changes PrivateTmp=disconnected as the following:
- If DefaultDependencies=no and RootDirectory=/RootImage= are not set,
then a private tmpfs is mounted _only_ on /tmp/, and set $TMPDIR=/tmp
environment variable to suggest the service to use /tmp/.
- If DefaultDependencies=yes and RootDirectory=/RootImage= are not set,
then implies RequiresMountsFor=/var/, though that is typically
redundant, but anyway. Hence, we can safely mount /var/tmp/.
- Otherwise, i.e. when one of RootDirectory=/RootImage= is set, behaves
as the same as the previous, as the private root filesystem for the
service is explicitly prepared by the service manager, and we can
safely mount a private tmpfs on /var/tmp/ without any extra
dependencies.
Fixes#37258.
Co-authored-by: Mike Yuan <me@yhndnzj.com>
Currently there are various circular dependencies between headers
in core/. Let's get rid of these by making judicious use of forward
declarations and moving includes into implementation files instead of
having them in header files.
Getting rid of circular header includes simplifies the code and makes
various clang based tooling such as iwyu work much better on our code.
The most important change is getting rid of the manager.h include in
unit.h which is possible thanks to the previous commits. We also move
the OOMPolicy and StatusType enums to unit.h to remove the need for
other unit headers to include manager.h to get access to these enums.
These variables are assigned with set_put_strv(), hence
string_hash_ops_free is used.
This also drops unnecessary call of set_ensure_allocated() with wrong
hash ops.
This delegates one or more namespaces to the service. Concretely,
this setting influences in which order we unshare namespaces. Delegated
namespaces are unshared *after* the user namespace is unshared. Other
namespaces are unshared *before* the user namespace is unshared.
Fixes#35369
Migrating ProtectHostname to enum will set the stage for adding more
properties like ProtectHostname=private in future commits.
In addition, we add PrivateHostnameEx property to dbus API which uses
string instead of boolean.
This new setting allows unsharing the pid namespace in a unit. Because
you have to fork to get a process into a pid namespace, we fork in
systemd-executor to get into the new pid namespace. The parent then
sends the pid of the child process back to the manager and exits while
the child process continues on with the rest of exec_invoke() and then
executes the actual payload.
Communicating the child pid is done via a new pidref socket pair that is
set up on manager startup.
We unshare the PID namespace right before the mount namespace so we
mount procfs correctly. Note PrivatePIDs=yes always implies MountAPIVFS=yes
to mount procfs.
When running unprivileged in a user session, user namespace is set up first
to allow for PID namespace to be unshared. However, when running in
privileged mode, we unshare the user namespace last to ensure the user
namespace does not own the PID namespace and cannot break out of the sandbox.
Note we disallow Type=forking services from using PrivatePIDs=yes since the
init proess inside the PID namespace must not exit for other processes in
the namespace to exist.
Note Daan De Meyer did the original work for this commit with Ryan Wilson
addressing follow-ups.
Co-authored-by: Daan De Meyer <daan.j.demeyer@gmail.com>
When an exec directory is shared between services, this allows one of the
service to be the producer of files, and the other the consumer, without
letting the consumer modify the shared files.
This will be especially useful in conjunction with id-mapped exec directories
so that fully sandboxed services can share directories in one direction, safely.
This commit refactors ProtectControlGroups= from using a boolean
in the dbus/execute backend to using an enum. There is no functional
change but this will allow adding new non-boolean values (e.g. strict,
private) a la PrivateHome.
This will allow units (scopes/slices/services) to override the default
systemd-oomd setting DefaultMemoryPressureDurationSec=.
The semantics of ManagedOOMMemoryPressureDurationSec= are:
- If >= 1 second, overrides DefaultMemoryPressureDurationSec= from oomd.conf
- If is empty, uses DefaultMemoryPressureDurationSec= from oomd.conf
- Ignored if ManagedOOMMemoryPressure= is not "kill"
- Disallowed if < 1 second
Note the corresponding dbus property is DefaultMemoryPressureDurationUSec
which is in microseconds. This is consistent with other time-based
dbus properties.
This adds the ExtraFileDescriptor property to StartTransient dbus API
with format "a(hs)" - array of (file descriptor, name) pairs. The FD
will be passed to the unit via sd_notify like Socket and OpenFile.
systemctl show also shows ExtraFileDescriptorName for these transient
units. We only show the name passed to dbus as the FD numbers will
change once passed over the unix socket and are duplicated, so its
confusing to display the numbers.
We do not add this functionality for systemd-run or general systemd
service units as it is not useful for general systemd services.
Arguably, it could be useful for systemd-run in bash scripts but we
prefer to be cautious and not expose the API yet.
Fixes: #34396
One of the major pait points of managing fleets of headless nodes is
that when something fails at startup, unless debug level was already
enabled (which usually isn't, as it's a firehose), one needs to manually
enable it and pray the issue can be reproduced, which often is really
hard and time consuming, just to get extra info. Usually the extra log
messages are enough to triage an issue.
This new option makes it so that when a service fails and is restarted
due to Restart=, log level for that unit is set to debug, so that all
setup code in pid1 and sd-executor logs at debug level, and also a new
DEBUG_INVOCATION=1 env var is passed to the service itself, so that it
knows it should start with a higher log level. Once the unit succeeds
or reaches the rate limit the original level is restored.
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
Lennart Poettering
Mike Yuan
Yu Watanabe
Usama Arif
Luiz Amaral
Luca Boccassi
Lennart Poettering
Quentin Deslandes
Daan De Meyer
Matteo Croce
Andres Beltran
Ryan Wilson