This commit adds a simple version of io.systemd.Unit.StartTransient
for varlink. It is similar to the dbus version, but there is a key
difference:
1. Instead of building the unit from key/value properties it
takes a json object in the "service" parameter. It is also
only implementing a minimal set of what can be done with a
service (for now)
2. No aux units (for now)
3. When called with --more the varlink socket can notify about
unit job and state changes controlled via a bool on the
varlink call inputs: notify{Job,Unit}Changes
We use the new io.systemd.Job interface when outputing the
io.systemd.Unit.StartTransient result as it makes the output
nice and mirrors the input.
Note that the property names follow the D-Bus naming to make a
future "systemctl show" transition from D-Bus -> varlink easier.
Because UnitContext is now also used for the inputs we need
to make a bunch of fields `SD_VARLINK_NULLABLE` so that the
input is even accepted. This does not affect the output, it
is still fully populated, just the schema. The ID of UnitContext
is still required.
Thanks to ikruglov and Lennart for their excellent feedback on
this.
So far when a job completed we'd never transition into any new state,
we'd just do some final processing work (such as notifying clients) and
destroy it.
Let's change that, and briefly enter a final state: "finished". This is
useful so that code that notifies clients can generically send the
quadruplet of id, type, state, result for any change notification and
naturally can communicate job completion that way: by setting the state
field to "finished".
Prompted by the discussions in: #41583
This changes the .result field to invalid initially, which arguably
makes more sense than "done", which was previously the default.
This is a correctnes fix, and afaics has no effect on the API, since we
do not expose this 1:1 as D-Bus property: it's only seen on D-Bus as
part of the job completion signal, at which part it is correctly
initialized.
Noticed while reviewing: https://github.com/systemd/systemd/pull/41583
Let's introduce these for the following two reasons:
- Using them makes sure the parameter names match between declaration
and definition (if both use the corresponding macros).
- They make sure developers can't forget the _const_ and _pure_
attributes for the declarations.
This commit also includes some include sorting fixes
The impl is directly visible to the compiler so it can apply
all sorts of optimizations wherever it sees fit. And with
the previous commit they are actually "const".
Currently there are various circular dependencies between headers
in core/. Let's get rid of these by making judicious use of forward
declarations and moving includes into implementation files instead of
having them in header files.
Getting rid of circular header includes simplifies the code and makes
various clang based tooling such as iwyu work much better on our code.
The most important change is getting rid of the manager.h include in
unit.h which is possible thanks to the previous commits. We also move
the OOMPolicy and StatusType enums to unit.h to remove the need for
other unit headers to include manager.h to get access to these enums.
Previously, unit_{start,stop,reload} would call the low-level cgroup
unfreeze function whenever a unit was started, stopped, or reloaded. It
did so with no error checking. This call would ultimately recurse up the
cgroup tree, and unfreeze all the parent cgroups of the unit, unless an
error occurred (in which case I have no idea what would happen...)
After the freeze/thaw rework in a previous commit, this can no longer
work. If we recursively thaw the parent cgroups of the unit, there may
be sibling units marked as PARENT_FROZEN which will no longer actually
have frozen parents. Fixing this is a lot more complicated than simply
disallowing start/stop/reload on a frozen unit
Fixes https://github.com/systemd/systemd/issues/15849
Follow-up for 2a39b91459
The mentioned change makes all jobs in the transaction unmergeable
if mode == JOB_RESTART_DEPENDENCIES, but we only want the anchor job
to be re-enqueued.
With the fix done in PR28215, the unit restart job is created with type JOB_START.
Because of that, it is not properly merged anymore with the old one: the
merged job has state JOB_RUNNING. It should have state JOB_WAITING.
I think that the old job is not cleaned up because we don't go through the failed state.
With this fix, the merged job is properly created with state JOB_WAITING.
This new job mode will enqueue a start job for a unit, and all units
depending on the unit will get a restart job enqueued. This is then used
for automatic sevice restarts: the unit itself is only started, the
depending units restarted. This way the unit will not go down
unnecessarily, triggering OnSuccess= needlessly.
This also introduces a new state SERVICE_AUTO_RESTART_QUEUED that is
entered once the restart jobs are enqueued. Previously we'd stay in
SERVICE_AUTO_RESTART, but that's problematic, since we'd lose
information whether we still need to enqueue the restart job during a
serialization/deserialization cycle or not. By having an explicit state
for this we know exactly whether we still need to enqueue the job or
not. It's also good since when we are in SERVICE_AUTO_RESTART_QUEUED we
want to act on unit_start(), but on SERVICE_AUTO_RESTART we want to wait
for the holdoff time to pass before we act on unit_start().
Fixes: #27722
Often I end up debugging a problem on a system, and I
do e.g. `journalctl --grep=failed|error`. The use of the term
"failed" for condition checks adds a *lot* of unnecessary noise into
this.
Now, I know this regexp search isn't precise, but it has proven
to be useful to me.
I think "failed" is too strong of a term as a baseline, and also
just stands out to e.g. humans watching their servers boot or
whatever.
The term "met condition" is fairly widely used, e.g.
https://stackoverflow.com/questions/63751794/what-does-the-condition-is-met-exactly-mean-in-programming-languages
Use that instead.
Not wired in by any unit type yet, just the basic to allocate,
ref, deref and plug in to other unit types.
Includes recording the trigger unit name and passing it to the
triggered unit as TRIGGER_UNIT= env var.
Remove the list logic, and simply skip passing metadata if more than one
unit triggered an OnFailure/OnSuccess handler.
Instead of a single env var to loop over, provide each separate item
as its own variable.
Fixes https://github.com/systemd/systemd/issues/22370
When a service exits and triggers either an OnFailure= or OnSuccess=
dependency we now set a new environment variable for the ExecStart= and
ExecStartPre= process. This variable $MONITOR_METADATA exposes the
metadata relating to the service which triggered the dependency.
MONITOR_METADATA takes the following form:
MONITOR_METADATA="SERVICE_RESULT=<result-string0>,EXIT_CODE=<exit-code0>,EXIT_STATUS=<exit-status0>,INVOCATION_ID=<id>,UNIT=<triggering-unit0.service>;SERVICE_RESULT=<result-stringN>,EXIT_CODE=<exit-codeN>,=EXIT_STATUS=<exit-statusN>,INVOCATION_ID=<id>,UNIT=<triggering-unitN.service>"
MONITOR_METADATA is space separated set of metadata relating to the
service(s) which triggered the dependency. This is a list since if we
have 2 services which trigger the same dependency then the dependency
start job may be merged. In this case we need to pass both service
metadata to the triggered service. If there is no job merging then
MONITOR_METADATA will be a single entry.
For example, in the case we had a service "failer.service" which
triggers "failer-handler.service", the following variable is exported to
the ExecStart= and ExecStartPre= processes in failer-handler.service:
MONITOR_METADATA="SERVICE_RESULT=exit-code,EXIT_CODE=exited,EXIT_STATUS=1,INVOCATION_ID=67c657ed7b34466ea369abdf994c6393,UNIT=failer.service"
In another example where we have failer.service and failer2.service
which both also trigger failer-handler.service then the start job for
failer-handler.service may be merged and we might get the following:
MONITOR_METADATA="SERVICE_RESULT=exit-code,EXIT_CODE=exited,EXIT_STATUS=1,INVOCATION_ID=16a93ad196c94109990fb8b9aa5eef5f,UNIT=failer.service;SERVICE_RESULT=exit-code,EXIT_CODE=exited,EXIT_STATUS=1,INVOCATION_ID=ff70131e4cc145e994fb621de25a3e8f,UNIT=failer2.service"
When an activation job is skipped because of a Condition*= setting failing,
currently the JobRemoved signal lists 'done' as the result, just as with
a successful job.
This is a problem when doing dbus activation: dbus-broker will receive a
signal that says the job was successful, so then it moves into a state where
it waits for the requested name to appear on the bus, but that never happens
because the job was actually skipped.
Add a new StartUnitWithFlags that changes the behaviour of the JobRemoved
signal to list 'done' or 'skipped'.
Fixes#21520
As suggested in https://github.com/systemd/systemd/pull/11484#issuecomment-775288617.
This does not touch anything exposed in src/systemd. Changing the defines there
would be a compatibility break.
Note that tests are broken after this commit. They will be fixed in the next one.
This reverts commit 097537f07a.
At least Fedora and Debian have already reverted this at the distro
level because it causes more problems than it solves. Arch is debating
reverting it as well [0] but would strongly prefer that this happens
upstream first. Fixes#15188.
[0] https://bugs.archlinux.org/task/66458
In the steps given in #13850, the resulting graph looks like:
C (Anchor) -> B -> A
Since B is inactive, it will be flagged as redundant and removed from
the transaction, causing A to get garbage collected. The proposed fix is
to not mark nodes as redundant if doing so causes a relevant node to be
garbage collected.
Fixes#13850
Jobs are added to the run queue in random order. This happens because most
jobs are added by iterating over the transaction or dependency hash maps.
As a result, jobs that can be executed at the same time are started in a
different order each time.
On small embedded devices this can cause a measurable jitter for the point
in time when a job starts (~100ms jitter for 10 units that are started in
random order).
This results is a similar jitter for the boot time. This is undesirable in
general and make optimizing the boot time a lot harder.
Also, jobs that should have a higher priority because the unit has a higher
CPU weight might get executed later than others.
Fix this by turning the job run_queue into a Prioq and sort by the
following criteria (use the next if the values are equal):
- CPU weight
- nice level
- unit type
- unit name
The last one is just there for deterministic sorting to avoid any jitter.
The job ordering logic is spread at multiple places of the code, making
it hard to maintain and also a bit to understand. The actual execution
order of two jobs always depends on their types and the ordering
contraint between their units. Extract this logic to a new function
job_compare. The second change is simplification of the order
evaluation, JOB_STOP takes always precedence (as documented), unless two
units are both stopping, then the ordering constraint is taken into
account.
Let's add a helper call unit_deserialize_job() for this purpose, and
let's move registration in the global jobs hash table into
job_install_deserialized() so that it it is done after all superficial
checks are done, and before transitioning into installed states, so that
rollback code is not necessary anymore.
Memory management is borked for this, and moreover this is unnecessary
since f0831ed2a0, i.e. since coldplug() and catchup() are two different
concepts: the former restoring the state from before a reload, the
latter than adjusting it again to the actual status in effect after the
reload.
Fixes: #10716
Mostly reverts: #8803
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
When "systemctl daemon-reload" is run at the same time as "systemctl
start foo", the latter might hang. That's because commands like start
wait for JobRemoved signal to know when the job is finished. But if the
job is finished during reloading, the signal is never sent.
The hang can be easily reproduced by running
# for ((N=1; N>0; N++)) ; do echo $N ; systemctl daemon-reload ; done
# for ((N=1; N>0; N++)) ; do echo $N ; systemctl start systemd-coredump.socket ; done
in two different terminals. The start command will hang after 1-2
iterations.
This keeps track of jobs that were started before reload and finished
during it and sends JobRemoved after the reload has finished.
Scope units are populated from PIDs specified by the bus client. We do
that when a scope is started. We really shouldn't allow scopes to be
started multiple times, as the PIDs then might be heavily out of date.
Moreover, clients should have the guarantee that any scope they allocate
has a clear runtime cycle which is not repetitive.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
This is a fixup of commit a2df3ea4ae.
When there is a running job with JobRunningTimeoutSec= and systemd serializes
its state (e.g. during daemon-reload), the timer event source won't be properly
restored in job_coldplug().
Thus save and serialize begin_running_usec too and reinitialize the timer based
on that value.
Michael Vogt
Lennart Poettering
Daan De Meyer
Mike Yuan
Lennart Poettering
Gavin Li
Zbigniew Jędrzejewski-Szmek
Adrian Vovk
Richard Phibel
Yu Watanabe
Colin Walters
Luca Boccassi
Peter Morrow
Luca Boccassi
Dave Reisner
Kevin Kuehler
Michael Olbrich
Anita Zhang
Michal Koutný
Ben Boeckel
David Tardon
Michal Koutný