The calculation for used memory in oomd_system_context_acquire is given
by MemTotal - MemFree from /proc/meminfo. This is too strict of a
calculation because it does not consider memory that is still available
for starting new applictions without swapping (MemAvailable). As a
result, systemd-oomd can start to kill processes before it is necessary.
This is more apparent on systems with low swap space.
Instead, compute 'used' memory as MemTotal - MemAvailable in
oomd_system_context_acquire and procfs_memory_get (which is used by
oomd_cgroup_context_acquire). And, rename oomd_mem_free_below to
oomd_mem_available_below for clarity.
To notify user of kill events from systemd-oomd we now use
`SERVICE_FAILURE_OOM_KILL` as the failure result.
`unit_check_oomd_kill` now calls `notify_cgroup_oom` to
update the service result to `oom-kill`.
We add a new xattr `user.oomd_ooms` to keep track of the OOM kills
initiated by systemd-oomd, this helps us resolve a race between sending
SIGKILL to processes and checking for OOM kill status from the xattr.
Related to: #20649
The approach to use '''…'''.split() instead of a list of strings was initially
used when converting from automake because it allowed identical blocks of lines
to be used for both, making the conversion easier.
But over the years we have been using normal lists more and more, especially
when there were just a few filenames listed. This converts the rest.
No functional change.
This is a follow up to 29f4185a9c ("oomd: Dump top offenders after a
kill action") to clean up the code a bit for review comments that
happened after the code had been merged already.
Currently if systemd-oomd doesn't kill anything in a selected cgroup, it
selects a new candidate immediately. But if a selected cgroup wasn't killed,
it is likely due to it disappearing or getting cleaned up between the time
it was selected as a candidate and getting sent SIGKILL(s). We should handle
it as though systemd-oomd did perform a kill so that it will check
swap/pressure again before it tries to select a new candidate.
There can be a situation where systemd-oomd would kill all of the processes
in a cgroup, pid1 would clean up that cgroup, and systemd-oomd would get
ENODEV trying to iterate the cgroup a final time to ensure it was empty.
systemd-oomd sees this as an error and immediately picks a new candidate even
though pressure may have recovered. To counter this, check and handle
path unavailability errnos specially.
Fixes: #22030
loadavg.h is an internal header of the Linux source repository, and as
such it is licensed as GPLv2-only, without syscall exception.
We use it only for 4 macros, which are simply doing some math calculations
that cannot thus be subject to copyright.
Reimplement the same calculations in another internal header and delete
loadavg.h from our tree.
Compared to PID1 where systemd-oomd has to be the client to PID1
because PID1 is a more privileged process than systemd-oomd, systemd-oomd
is the more privileged process compared to a user manager so we have
user managers be the client whereas systemd-oomd is now the server.
The same varlink protocol is used between user managers and systemd-oomd
to deliver ManagedOOM property updates. systemd-oomd now sets up a varlink
server that user managers connect to to send ManagedOOM property updates.
We also add extra validation to make sure that non-root senders don't
send updates for cgroups they don't own.
The integration test was extended to repeat the chill/bloat test using
a user manager instead of PID1.
Gets rid of a few gotos, allows removing the extra ret variable and
will also be used in a future commit by the codepath that receives
cgroups from user instances of systemd.
LLVM 13 introduced `-Wunused-but-set-variable` diagnostic flag, which
trips over some intentionally set-but-not-used variables or variables
attached to cleanup handlers with side effects (`_cleanup_umask_`,
`_cleanup_(notify_on_cleanup)`, `_cleanup_(restore_sigsetp)`, etc.):
```
../src/basic/process-util.c:1257:46: error: variable 'saved_ssp' set but not used [-Werror,-Wunused-but-set-variable]
_cleanup_(restore_sigsetp) sigset_t *saved_ssp = NULL;
^
1 error generated.
```
In general we almost never hit those asserts in production code, so users see
them very rarely, if ever. But either way, we just need something that users
can pass to the developers.
We have quite a few of those asserts, and some have fairly nice messages, but
many are like "WTF?" or "???" or "unexpected something". The error that is
printed includes the file location, and function name. In almost all functions
there's at most one assert, so the function name alone is enough to identify
the failure for a developer. So we don't get much extra from the message, and
we might just as well drop them.
Dropping them makes our code a tiny bit smaller, and most importantly, improves
development experience by making it easy to insert such an assert in the code
without thinking how to phrase the argument.
cb13961ada updated the oomd logic to
collect candidate data when a kill was about to happen. However there
was still a call left over in the main loop to collect candidate data on
every interval. Remove this since it's unneeded.
Fixes#20122
fd_duplicate_data_fd() is renamed to copy_data_fd(). This makes
the two functions have nicely similar names.
Now fd-util.[ch] is again about low-level file descriptor manipulations.
copy_data_fd() is a complex function that internally wraps the other
functions in copy.c. I want to move copy.c and the whole cluster of
related code from basic/ to shared/ later on, and this is a preparatory
step for that.
The kernel can be compiled without support for any memory.swap.* files, or
it can be disabled at boot time with the 'swapaccount=0' boot parameter,
so if the file doesn't exist log warning indicating the kernel doesn't
support the file and the user may need to try using the 'swapaccount=1'
boot param.
Note that the actual error from the call to fopen() is ENOENT, but
that is translated into ENODATA in cg_get_attribute_as_uint64()
In some instances, particularly with swap on zram, swap used will be high
while there is still a lot of memory available. FB OOMD handles this by
thresholding kills to X% of total swap usage. Let's do the same thing here.
Anecdotally with these thresholds and my laptop which is exclusively swap
on zram I can sit at 0K / 4G swap free with most of memory free and
systemd-oomd doesn't kill anything.
Partially addresses aggressive kill behavior from
https://bugzilla.redhat.com/show_bug.cgi?id=1941170
When this test is run in mkosi, the previously tested cgroup that we write
xattrs into and the root cgroup are the same.
Since the root cgroup is a live cgroup anyways (vs. the test cgroups which are
remade each time) let's generate the expected preference values from reading
the xattrs instead of assuming it will be NONE.
Since this is only changed the first time the limit is hit (and remains
set as long as the pressure remains over), I changed the name to better
reflect that.
Keeps consistent with "last_had_mem_reclaim" which is actually updated
every time there is reclaim activity.
Yu Watanabe
Michael Biebl
Zbigniew Jędrzejewski-Szmek
Nick Rosbrook
Nishal Kulkarni
Benjamin Berg
Anita Zhang
Jan Janssen
Luca Boccassi
Daan De Meyer
Frantisek Sumsal
Dan Streetman
Jóhann B. Guðmundsson