service: add 'debug' option to RestartMode=

One of the major pait points of managing fleets of headless nodes is
that when something fails at startup, unless debug level was already
enabled (which usually isn't, as it's a firehose), one needs to manually
enable it and pray the issue can be reproduced, which often is really
hard and time consuming, just to get extra info. Usually the extra log
messages are enough to triage an issue.

This new option makes it so that when a service fails and is restarted
due to Restart=, log level for that unit is set to debug, so that all
setup code in pid1 and sd-executor logs at debug level, and also a new
DEBUG_INVOCATION=1 env var is passed to the service itself, so that it
knows it should start with a higher log level. Once the unit succeeds
or reaches the rate limit the original level is restored.
This commit is contained in:
Luca Boccassi
2024-08-27 12:24:45 +01:00
committed by Luca Boccassi
parent 23e3bddc48
commit 7d8bbfbe08
13 changed files with 163 additions and 23 deletions
+6 -2
View File
@@ -462,6 +462,8 @@ struct ExecParameters {
char *unit_id;
sd_id128_t invocation_id;
char invocation_id_string[SD_ID128_STRING_MAX];
bool debug_invocation;
};
#define EXEC_PARAMETERS_INIT(_flags) \
@@ -629,7 +631,8 @@ bool exec_needs_ipc_namespace(const ExecContext *context);
const ExecContext *_c = (ec); \
const ExecParameters *_p = (ep); \
const int _l = (level); \
bool _do_log = _c->log_level_max < 0 || \
bool _do_log = _p->debug_invocation || \
_c->log_level_max < 0 || \
_c->log_level_max >= LOG_PRI(_l); \
LOG_CONTEXT_PUSH_IOV(_c->log_extra_fields, \
_c->n_log_extra_fields); \
@@ -674,7 +677,8 @@ bool exec_needs_ipc_namespace(const ExecContext *context);
const ExecContext *_c = (ec); \
const ExecParameters *_p = (ep); \
const int _l = (level); \
bool _do_log = _c->log_level_max < 0 || \
bool _do_log = _p->debug_invocation || \
_c->log_level_max < 0 || \
_c->log_level_max >= LOG_PRI(_l); \
LOG_CONTEXT_PUSH_IOV(_c->log_extra_fields, \
_c->n_log_extra_fields); \