We get the same warning thousands of times:
/work/src/tools/check-version-history.py:28: FutureWarning: This search
incorrectly ignores the root element, and will be fixed in a future
version. If you rely on the current behaviour, change it to
"./refsynopsisdiv/funcsynopsis/funcprototype/funcdef/function[.='udev_device_get_properties_list_entry']"
We also need to update the ignorelist to the new form.
We get the same warning thousands of times:
/work/src/tools/check-version-history.py:28: FutureWarning: This search incorrectly
ignores the root element, and will be fixed in a future version. If you rely on the
current behaviour, change it to
"./refsynopsisdiv/funcsynopsis/funcprototype/funcdef/function[.='udev_device_get_properties_list_entry']"
We also need to update the ignorelist to the new form.
* 13d523f84d Relax dependencies from noarch packages on archful packages for OBS builds
* 59378485be Remove purge-nobody-user script
* d1380dc114 Add more services to %post for udev and networkd
* 6f0d03443d Fix paths for /usr/sbin/nologin and related progs
* df9a74d530 Make the source tarball glob in the test script more generic
Integration test units are now connected to the tty when running
interactively, so let's make sure we disable the pager to avoid tests
hanging in the pager.
This introduce `LOG_ITEM()` macro that checks format strings in
log_struct() and friends.
Hopefully, this silences false-positive warnings by Coverity.
If we're using these helpers, we want to split a user record into two,
one with the privileged section, and one without. This should work even
when the user record has a "status" section, so adapt the helpers to
account for that.
As reported in https://github.com/systemd/systemd/issues/35405, if the watchdog
ping failed, we effectively started a busy loop here. The previous commits
should fix this, but in general, the protection here is intended as a safety
net in case the logic is broken somewhere else. We shouldn't exclude the
watchdog stuff from this.
Closes https://github.com/systemd/systemd/issues/35405. Apparently some
watchdog devices can be opened, but then the pings start failing after some
time. Since the timestamp of the last successful ping is not updated, we try to
ping again immediately, causing a busy loop and excessive logging.
After trying a few different approaches to fit this into the existing framework
without changing the logic too much, I settled on an approach with a second
timestamp. In particular, the timestamp of the last successful ping is public,
exposed as WatchdogLastPingTimestamp over dbus. It'd be wrong to redefine this
to mean the last ping *attempt*. So we need a second timestamp in some form.
Also, if we give up on pinging, we probably should attempt to disarm the
watchdog. It's possible that the pinging fails, but the watchdog would still
fire. I don't think we want that, since it seems that our internal loop is
working, it's just the watchdog that is broken.
Structured message with SD_MESSAGE_WATCHDOG_PING_FAILED is logged if we fail
to ping.
I tested this by attaching gdb to pid 1 and calling close(watchdog_fd).
We get a bunch of warning messages and then an attempt to close the watchdog:
Mar 21 15:46:17 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:20 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:23 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:26 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:29 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:32 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:35 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:37 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:40 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:43 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:46 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:49 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:52 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:55 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0: Bad file descriptor
Mar 21 15:46:58 fedora systemd[1]: Failed to ping hardware watchdog /dev/watchdog0, closing watchdog after 15 attempts: Bad file descriptor
Mar 21 15:46:58 fedora systemd[1]: Failed to disable hardware watchdog, ignoring: Bad file descriptor
Mar 21 15:46:58 fedora systemd[1]: Failed to disarm watchdog timer, ignoring: Bad file descriptor
Maintaining the fmf metadata and script upstream makes it painful
to reuse downstream so let's move the metadata and testing script
downstream and load it upstream instead.
* 2ecfbec1a4 Support specifying extra mkosi repositories to the test script
* f5b47b1302 Use old setup sysusers files on Fedora < 43
* 2da5793357 Merge #196 `Migrate fmf metadata and test script from the upstream repository`
* e346d9f33e Limit sdubby dependency to Fedora
One ID for the success case, with WATCHDOG_DEVICE= showing the device,
and one ID for the failure case, with WATCHDOG_DEVICE= if configured,
and ERRNO= set automatically.
We need to retry the open attempts for the watchdog, because the device becomes
available asynchronously.
The watchdog is opened in two places:
- in pid1 in the main loop. The loop has a ratelimit, but during a boot we
iterate in it fairly quickly. On my test VM with 'iTCO_wdt', version 2:
$ journalctl -b --grep 'Failed to open any watchdog' | wc -l
3398
After the device has been processed by udev, it is initialized successfully.
- in shutdown. In that case, we most likely don't need to try more than once,
because we mostly care about the case where the watchdog device was present
and configured previously. But in principle it is possible that we might
attempt shutdown while the machine was initializing, so we don't want to
disable retries. Nevertheless, watchdog_ping() is called from a loop that
might be fairly tight, so we could end up trying to reopen the device fairly
often. This probably doesn't matter *too* much, but it's still ugly to try to
open the device without any ratelimit.
Usually the watchdog timeout would be set to something like 30 s or a few
minutes. OTOH, on my VM, the device becomes avaiable at 4.35 s after boot. So
let's use 5 s or half the watchdog timeout, whatever is smaller, as the
interval.
Without this change, when e.g. event_timeout= is specified in udev.conf,
udevadm and friends which loads udev.conf warn about unknown key:
===
$ udevadm info /sys/class/net/lo
/run/udev/udev.conf.d/test-17.conf:1: Unknown key 'event_timeout', ignoring.
/run/udev/udev.conf.d/test-17.conf:2: Unknown key 'timeout_signal', ignoring.
===
Follow-up for 07f5e35fe7 (v256).
I think we need to log at some point if the user configured a watchdog device,
but no devices were found. We can't log ENOENT immediately, because the device
may likely appear during boot. So wait until the end of the initial transaction
and log then.
If we failed to open the watchdog device for any reason, we'd only log at debug
level. This seems iffy: if the user configured a timeout for the watchdog, we
should report when we can't set it up. ENOENT is still logged at debug level
only, since it's somewhat expected to have a watchdog timeout set up, even for
systems which don't have a watchdog, or the device might appear later.
If the device doesn't support WDIOC_GETSUPPORT, still log that we opened a
device.
No change in behaviour, except for the log level threshold. As a side effect,
the reason why we failed to open the device is now stored in watchdog_fd
(previously it was -1 always), but this isn't used for anything.
This is not a hot path, but let's move the variables to the
minimal scope in which they are needed.
Also, all functions which call watchdog_open() check for fd < 0 first,
so change the conditional return into an assert.
Those functions call watchdog_setup() and watchdog_setup_pretimeout(), which
internally do a similar check against the static variables watchdog_timeout and
watchdog_pretimeout. The second check is not useful.
Daan De Meyer
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
Luca Boccassi
Busayo Dada
Yu Watanabe
Erik Larsson
Lennart Poettering
Jose Ortuno
Matteo Croce
Mike Yuan