As reported in https://github.com/systemd/systemd/issues/35405, if the watchdog
ping failed, we effectively started a busy loop here. The previous commits
should fix this, but in general, the protection here is intended as a safety
net in case the logic is broken somewhere else. We shouldn't exclude the
watchdog stuff from this.
Closes https://github.com/systemd/systemd/issues/35405. Apparently some
watchdog devices can be opened, but then the pings start failing after some
time. Since the timestamp of the last successful ping is not updated, we try to
ping again immediately, causing a busy loop and excessive logging.
After trying a few different approaches to fit this into the existing framework
without changing the logic too much, I settled on an approach with a second
timestamp. In particular, the timestamp of the last successful ping is public,
exposed as WatchdogLastPingTimestamp over dbus. It'd be wrong to redefine this
to mean the last ping *attempt*. So we need a second timestamp in some form.
Also, if we give up on pinging, we probably should attempt to disarm the
watchdog. It's possible that the pinging fails, but the watchdog would still
fire. I don't think we want that, since it seems that our internal loop is
working, it's just the watchdog that is broken.
Structured message with SD_MESSAGE_WATCHDOG_PING_FAILED is logged if we fail
to ping.
I tested this by attaching gdb to pid 1 and calling close(watchdog_fd).
We get a bunch of warning messages and then an attempt to close the watchdog:
Mar 21 15:46:17 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:20 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:23 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:26 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:29 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:32 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:35 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:37 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:40 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:43 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:46 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:49 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:52 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:55 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:58 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0, closing watchdog after 15 attempts: Bad file descriptor
Mar 21 15:46:58 fedora systemd[1]: Failed to disable hardware watchdog, ignoring: Bad file descriptor
Mar 21 15:46:58 fedora systemd[1]: Failed to disarm watchdog timer, ignoring: Bad file descriptor
One ID for the success case, with WATCHDOG_DEVICE= showing the device,
and one ID for the failure case, with WATCHDOG_DEVICE= if configured,
and ERRNO= set automatically.
We need to retry the open attempts for the watchdog, because the device becomes
available asynchronously.
The watchdog is opened in two places:
- in pid1 in the main loop. The loop has a ratelimit, but during a boot we
iterate in it fairly quickly. On my test VM with 'iTCO_wdt', version 2:
$ journalctl -b --grep 'Failed to open any watchdog' | wc -l
3398
After the device has been processed by udev, it is initialized successfully.
- in shutdown. In that case, we most likely don't need to try more than once,
because we mostly care about the case where the watchdog device was present
and configured previously. But in principle it is possible that we might
attempt shutdown while the machine was initializing, so we don't want to
disable retries. Nevertheless, watchdog_ping() is called from a loop that
might be fairly tight, so we could end up trying to reopen the device fairly
often. This probably doesn't matter *too* much, but it's still ugly to try to
open the device without any ratelimit.
Usually the watchdog timeout would be set to something like 30 s or a few
minutes. OTOH, on my VM, the device becomes avaiable at 4.35 s after boot. So
let's use 5 s or half the watchdog timeout, whatever is smaller, as the
interval.
I think we need to log at some point if the user configured a watchdog device,
but no devices were found. We can't log ENOENT immediately, because the device
may likely appear during boot. So wait until the end of the initial transaction
and log then.
If we failed to open the watchdog device for any reason, we'd only log at debug
level. This seems iffy: if the user configured a timeout for the watchdog, we
should report when we can't set it up. ENOENT is still logged at debug level
only, since it's somewhat expected to have a watchdog timeout set up, even for
systems which don't have a watchdog, or the device might appear later.
If the device doesn't support WDIOC_GETSUPPORT, still log that we opened a
device.
No change in behaviour, except for the log level threshold. As a side effect,
the reason why we failed to open the device is now stored in watchdog_fd
(previously it was -1 always), but this isn't used for anything.
This is not a hot path, but let's move the variables to the
minimal scope in which they are needed.
Also, all functions which call watchdog_open() check for fd < 0 first,
so change the conditional return into an assert.
Those functions call watchdog_setup() and watchdog_setup_pretimeout(), which
internally do a similar check against the static variables watchdog_timeout and
watchdog_pretimeout. The second check is not useful.
Let's allow providing extra userdb users and groups via credentials.
Similarly to systemd-udev-load-credentials.service, we ship
systemd-userdb-load-credentials.service which transform the JSON
user/group records provided via the corresponding credentials to static
userdb dropins in /run/userdb.
Let's allow providing extra userdb users and groups via credentials.
Similarly to systemd-udev-load-credentials.service, we ship
systemd-userdb-load-credentials.service which transform the JSON
user/group records provided via the corresponding credentials to static
userdb dropins in /etc/userdb.
Replaces #33811
Beta was just released, let's switch to Fedora 42 which coincidentally
also has a crucial fix for its nsswitch.conf to make the next commits
actually work.
This also makes the generator not trigger an assertion added by
1cd3c49d09. If getty.ttys.container
credential contains a line prefixed with '/dev/', then the assertion
assert(!path_startswith(tty, "/dev/"));
was triggered. This drops the offending assertion, and such lines
are handled gracefully now.
Also, an empty string, "/dev/", and "/dev/pts/" (that is, a directory
without tty name) are gracefully skipped now.
Let's return the size in a return parameter instead of the return value.
And if NULL is specified this tells us the caller doesn't care about the
size and expects a NUL terminated string. In that case look for an
embedded NUL byte, and refuse in that case.
This should lock things down a bit, as we'll systematically refuse
embedded NUL strings now when we expect strings.
PR #34685 moved the handling of keys 66/65 from specific models to
generic HP laptops.
Key 66 has been linked to the `pickup_phone` function; however, this
action key is not available on all HP laptop models, particularly older
versions. On my HP Pavilion laptop, key 66 is mapped to the `backspace`
function, which caused the backspace key to stop working after the
change.
The following PR fixes the issue on my **HP Pavilion Laptop 15-eg0xxx**.
I have placed the modifications under the Pavilion section, but I cannot
guarantee that this solution will apply to all Pavilion models.
Additionally, I have included a line that checks for "HP" instead of
solely searching for "Hewlett-Packard," as my model is simply labeled as
HP.
Follow-up for c8f59296bf
Currently, the unit is only reffed in transient_unit_set_properties()
via AddRef(), which however would be dropped if a reconnection
is attempted. Make sure to explicitly re-add reference in that case.
This extends nsresourced to also allow delegation of a network tap
device (in addition to veth) to unpriv clients, with a strictly enforced
naming scheme.
also tightens security on a couple of things:
* enforces polkit on all nsresourced ops too (though by default still
everything is allowed)
* put a limit on delegated network devices
* forcibly clean up delegated network devices when the userns goes away
userdb allows user/group records without UID/GID (it only really
requires a name), in order to permit "unfixated" records. But that means
we cannot just rely on the field to be valid. And we mostly got that
right, but not everywhere. Fix that.
Let's tighten rules on namespace operations: let's always ask PK for
permission before doing anything.
Note that if polkit is absent we'll still allow things, and the default
PK policy will also still allow things, but there's now a clear way how
people can not allow things if they want, by modifying the PK policy.
Let's tighten the screws a bit on the network interfaces we delegate,
and explicitly destroy them, just like we destroy delegated cgroups.
Ideally, this should happen automatically because the userns goes away
that pins the veth, or because the client holding an fd for a tap device
goes away as the userns goes away. But you never know who keeps a
reference, hence let's explicitly destroy this too.
Zbigniew Jędrzejewski-Szmek
Daan De Meyer
Lennart Poettering
Yu Watanabe
Sonia Zorba
Mike Yuan