There is currently no way to figure out a rate limit was hit on a unit,
as the last result is stripped in order to keep reporting the first
result, which is useful in case of a watchdog failure, which is the
reason why it was changed as such.
But rate limiting is also an important information to provide to
users, so allow the Result property to reflect it when it
happens.
After bdb8e584f4 we stopped rebasing the
next elapse timestamp unconditionally and the only case where we'd do
that was when both last trigger and last inactive timestamps were empty.
This covered timer units during boot just fine, since they would have
neither of those timestamps set. However, persistent timers
(Persistent=yes) store their last trigger timestamp on a persistent
storage and load it back after reboot, so the rebasing was skipped in
this case.
To mitigate this, check the last_trigger timestamp is older than the
current machine boot - if so, that means that it came from a stamp file
of a persistent timer unit and we need to rebase it to make
RandomizedDelaySec= work properly.
Follow-up for bdb8e584f4.
Resolves: #39739
Otherwise, e.g. requesting to start a unit that is under stopping may
enter the failed state.
This makes
- rename .can_start() -> .test_startable(), and make it allow to return
boolean and refuse to start units when it returns false,
- refuse earlier to start units that are in the deactivating state, so
several redundant conditions in .start() can be dropped,
- move checks for unit states mapped to UNIT_ACTIVATING from .start() to
.test_startable().
Fixes#39247.
The test added in f4c3c107d9 uncovered a
corner case while recalculating the next elapse timestamp of a timer unit
that uses RandomizedDelaySec= during deserialization.
If the scheduled time (without RandomizedDelaySec=) already elapsed,
systemd "rebases" the next elapse timestamp to the time when systemd
first started, to make the RandomizedDelaySec= feature work even at
boot. However, since it was done unconditionally, it always overrode the
next elapse timestamp, which could then cause the final next elapse
timestamp to fall out of the expected window.
With a couple of additional debug logs one of the test fail looks like
this:
[ 132.129815] TEST-53-TIMER.sh[384]: + : 'Next elapse timestamp after daemon-reload, try #328'
[ 132.129815] TEST-53-TIMER.sh[384]: + systemctl daemon-reload
[ 132.136352] systemd[1]: Reload requested from client PID 16399 ('systemctl') (unit TEST-53-TIMER.service)...
[ 132.136636] systemd[1]: Reloading...
[ 132.446160] systemd[1]: Rebasing next elapse timestamp
[ 132.446168] systemd[1]: v->next_elapse: Tue 2025-10-14 00:10:00 CEST
[ 132.446170] systemd[1]: rebased: Tue 2025-10-14 00:10:56 CEST
[ 132.446172] systemd[1]: v->next_elapse after rebase: Tue 2025-10-14 00:10:56 CEST
[ 132.447361] systemd[1]: Reloading finished in 310 ms.
[ 132.484041] TEST-53-TIMER.sh[384]: + check_elapse_timestamp
[ 132.484041] TEST-53-TIMER.sh[384]: + systemctl status timer-RandomizedDelaySec-16377.timer
[ 132.533657] TEST-53-TIMER.sh[16440]: ● timer-RandomizedDelaySec-16377.timer
[ 132.533657] TEST-53-TIMER.sh[16440]: Loaded: loaded (/run/systemd/system/timer-RandomizedDelaySec-16377.timer; static)
[ 132.533657] TEST-53-TIMER.sh[16440]: Active: active (waiting) since Mon 2025-10-13 23:00:00 CEST; 1h 13min ago
[ 132.533657] TEST-53-TIMER.sh[16440]: Invocation: 5555d4f060114a5493ff228013830d17
[ 132.533657] TEST-53-TIMER.sh[16440]: Trigger: Tue 2025-10-14 22:10:04 CEST; 21h left
[ 132.533657] TEST-53-TIMER.sh[16440]: Triggers: ● timer-RandomizedDelaySec-16377.service
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:07 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Changed dead -> waiting
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:07 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Adding 15h 35min 1.230173s random time.
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:07 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Realtime timer elapses at Tue 2025-10-14 15:45:58 CEST.
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:07 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Changed dead -> waiting
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:08 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Adding 16h 29min 44.084409s random time.
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:08 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Realtime timer elapses at Tue 2025-10-14 16:40:41 CEST.
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:08 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Changed dead -> waiting
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:08 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Adding 21h 59min 7.955828s random time.
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:08 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Realtime timer elapses at Tue 2025-10-14 22:10:04 CEST.
[ 132.533657] TEST-53-TIMER.sh[16440]: Oct 14 00:13:08 H systemd[1]: timer-RandomizedDelaySec-16377.timer: Changed dead -> waiting
[ 132.535386] TEST-53-TIMER.sh[384]: + systemctl show -p InactiveExitTimestamp timer-RandomizedDelaySec-16377.timer
[ 132.537727] TEST-53-TIMER.sh[16442]: InactiveExitTimestamp=Mon 2025-10-13 23:00:00 CEST
[ 132.540317] TEST-53-TIMER.sh[16444]: ++ systemctl show -P NextElapseUSecRealtime timer-RandomizedDelaySec-16377.timer
[ 132.547745] TEST-53-TIMER.sh[384]: + NEXT_ELAPSE_REALTIME='Tue 2025-10-14 22:10:04 CEST'
[ 132.548020] TEST-53-TIMER.sh[16445]: ++ date '--date=Tue 2025-10-14 22:10:04 CEST' +%s
[ 132.550218] TEST-53-TIMER.sh[384]: + NEXT_ELAPSE_REALTIME_S=1760472604
[ 132.550218] TEST-53-TIMER.sh[384]: + : 'Next elapse timestamp should be Tue 2025-10-14 00:10:00 CEST <= Tue 2025-10-14 22:10:04 CEST <= Tue 2025-10-14 22:10:00 CEST'
[ 132.550218] TEST-53-TIMER.sh[384]: + assert_ge 1760472604 1760393400
[ 132.550555] TEST-53-TIMER.sh[16446]: + set +ex
[ 132.550702] TEST-53-TIMER.sh[384]: + assert_le 1760472604 1760472600
[ 132.550832] TEST-53-TIMER.sh[16447]: + set +ex
[ 132.551091] TEST-53-TIMER.sh[16447]: FAIL: '1760472604' > '1760472600'
Here the original next elapse timestamp was Tue 2025-10-14 00:10:00 CEST
as expected, but it was overridden by the rebased timestamp:
Tue 2025-10-14 00:10:56 CEST. And when a new randomized delay was added
to it (21h 59min 7.955828s) the final next elapse timestamp fell out of
the expected window, i.e. Tue 2025-10-14 00:10:00 (scheduled time) <
Tue 2025-10-14 22:10:04 CEST (rebased elapse timestamp + randomized
delay) < Tue 2025-10-14 22:10:00 CEST (scheduled time + maximum from
RandomizedDelaySec=, i.e. 22h).
By limiting the timestamp rebase only the case where the unit hasn't
already run should prevent this from happening during daemon-reload.
The only clock change that affects monotonic timers is suspend causing
CLOCK_MONOTONIC to fall behind CLOCK_BOOTTIME. However, monotonic timers
that use CLOCK_BOOTTIME are not intended to have their elapse time
changed after a suspend. Hence condition the calculation of monotonic
timers’ elapse times on the cause not being a clock change.
Fixes#29245
Currently there are various circular dependencies between headers
in core/. Let's get rid of these by making judicious use of forward
declarations and moving includes into implementation files instead of
having them in header files.
Getting rid of circular header includes simplifies the code and makes
various clang based tooling such as iwyu work much better on our code.
The most important change is getting rid of the manager.h include in
unit.h which is possible thanks to the previous commits. We also move
the OOMPolicy and StatusType enums to unit.h to remove the need for
other unit headers to include manager.h to get access to these enums.
By default, in instances where timers are running on a realtime schedule,
if a service takes longer to run than the interval of a timer, the
service will immediately start again when the previous invocation finishes.
This is caused by the fact that the next elapse is calculated based on
the last trigger time, which, combined with the fact that the interval
is shorter than the runtime of the service, causes that elapse to be in
the past, which in turn means the timer will trigger as soon as the
service finishes running.
This behavior can be changed by enabling the new DeferReactivation setting,
which will cause the next calendar elapse to be calculated based on when
the trigger unit enters inactivity, rather than the last trigger time.
Thus, if a timer is on an realtime interval, the trigger will always
adhere to that specified interval.
E.g. if you have a timer that runs on a minutely interval, the setting
guarantees that triggers will happen at *:*:00 times, whereas by default
this may skew depending on how long the service runs.
Co-authored-by: Matteo Croce <teknoraver@meta.com>
This is the equivalent of RequiresMountsFor=, but adds Wants= instead
of Requires=. It will be useful for example for the autogenerated
systemd-cryptsetup units.
Fixes https://github.com/systemd/systemd/issues/11646
Those functions take a pointer to a timestamp and return a timestamp pointer,
so the reader would be justified to think that those are just getters. Rename
them to avoid confusion.
This adds a new structure UnitDefaults which embedds the various default
settings for units we maintain. We so far maintained two sets of
variables for this, one in main.c as static variables and one in the
Manager structure. This moves them into a common structure.
This is most just search/replace, i.e. very dumb refactoring.
The fact that we now use a common structure for this allows us further
refactorings later.
Inspired by the discussions on #27890
This is supposed to be a help for compilers to apply optimizations on
functions where they can't determine whether they are const/pure on
their own. For static, local functions the compiler can do this on its
own easily however, hence the decoration with pure/const is just noise.
Let's drop it, and let the compiler to its thing better.
(Use it for exported functions, since compilers can't 'reach-over' into
other modules to determine if they are pure, except if LTO is used)
As it says on the tin, configures the unit to survive a soft reboot.
Currently all the following options have to be set by hand:
Conflicts=reboot.target kexec.target poweroff.target halt.target
Before=reboot.target kexec.target poweroff.target halt.target
After=sysinit.target basic.target
DefaultDependencies=no
IgnoreOnIsolate=yes
This is not very user friendly. If new default dependencies are added,
or new shutdown/reboot types, they also have to be added manually.
The new option is much simpler, easy to find, and does the right thing
by default.
If we're doing a daemon-reload, we'll be going from TIMER_DEAD => TIMER_WAITING,
so we won't use inactive_exit_timestamp because TIMER_DEAD != UNIT_ACTIVE, even
though inactive_exit_timestamp is serialized/deserialized and will be valid after
the daemon-reload.
This issue can lead to timers never firing as we'll always calculate the next
elapse based on the current realtime on daemon-reload, so if daemon-reload happens
often enough, the elapse interval will be moved into the future every time, which
means the timer will never trigger.
To fix the issue, let's always use inactive_exit_timestamp if it is set, and only
fall back to the current realtime if it is not set.
Now that we have a potentially pinned fdstore let's add a concept for
cleaning it explicitly on user requested. Let's expose this via
"systemctl clean", i.e. the same way as user directories are cleaned.
This fix unfortunately introduced a much worse regression that
is affecting many users, so let's revert it for now and rework
it in the next release.
This reverts commit 5ded3917a1.
Fixes https://github.com/systemd/systemd/issues/24984
(s) is just ugly with a vibe of DOS. In most cases just using the normal plural
form is more natural and gramatically correct.
There are some log_debug() statements left, and texts in foreign licenses or
headers. Those are not touched on purpose.
Same idea as 03677889f0.
No functional change intended. The type of the iterator is generally changed to
be 'const char*' instead of 'char*'. Despite the type commonly used, modifying
the string was not allowed.
I adjusted the naming of some short variables for clarity and reduced the scope
of some variable declarations in code that was being touched anyway.
Up until now the main reason why we didn't proceed with starting the
unit was exceed start limit burst. However, for unit types like mounts
the other reason could be effective ratelimit on /proc/self/mountinfo
event source. That means our mount unit state may not reflect current
kernel state. Hence, we need to attempt to re-run the start job again
after ratelimit on event source expires.
As we will be introducing another reason than start limit let's rename
the virtual function that implements the check.
Alternative to https://github.com/systemd/systemd/pull/20531.
Whenever a service triggered by another unit fails condition checks,
stop the triggering unit to prevent systemd busy looping trying to
start the triggered unit.
Fixes#17433. Currently, if any of the validations we do before we
check start rate limiting fail, we can still enter a busy loop as
no rate limiting gets applied. A common occurence of this scenario
is path units triggering a service that fails a condition check.
To fix the issue, we simply move up start rate limiting checks to
be the first thing we do when starting a unit. To achieve this,
we add a new method to the unit vtable and implement it for the
relevant unit types so that we can do the start rate limit checks
earlier on.
In general we almost never hit those asserts in production code, so users see
them very rarely, if ever. But either way, we just need something that users
can pass to the developers.
We have quite a few of those asserts, and some have fairly nice messages, but
many are like "WTF?" or "???" or "unexpected something". The error that is
printed includes the file location, and function name. In almost all functions
there's at most one assert, so the function name alone is enough to identify
the failure for a developer. So we don't get much extra from the message, and
we might just as well drop them.
Dropping them makes our code a tiny bit smaller, and most importantly, improves
development experience by making it easy to insert such an assert in the code
without thinking how to phrase the argument.
Mike Yuan
Luca Boccassi
Yu Watanabe
Frantisek Sumsal
Lukas Nykryn
Fleuria
Lennart Poettering
Daan De Meyer
Adrian Vovk
Arthur ShauandMatteo Croce
Zbigniew Jędrzejewski-Szmek
Luca Boccassi
David Tardon
João Loureiro
Michal Sekletar