I am really not a fan of full code lines passed to macros as parameters.
Let's get rid of the 3rd parameter of FOREACH_OPTION() hence:
1. Let's return errors just as a regular value (though a negative one),
that can be handled via a OPTION_ERROR case statement for the switch.
This normalizes handling of the error, just like any other event
returned by the option parser.
2. In order to avoid exploding the amount of boilerplate in each use
(that just propagates the error on OPTION_ERROR), let's then
introduce an explicit FOREACH_OPTION_OR_RETURN(), that returns from
the calling function on its own (and makes that clear in the name).
Together this cleans up, normalizes the logic and shortens the code.
It was requested to make the 'c', 'opt', and 'arg' params the same, i.e.
defined through the FOREACH_OPTION macro. But we can't do that easily,
because 'c' was defined in the for loop definition, and we can only
define variables of the same type in that way. Also, in some cases we
need only 'c', in other cases with need 'c' and 'arg, in some cases 'c'
and 'opt', and in other cases all three. We'd need to either
conditionalize or mark those variables with _unused_ to deal with
compiler warnings. But a different approach works quite nicely: add
state.opt and state.arg to show the current option and it's argument.
(The short names are picked on purpose to reduce verbosity since those
are used a lot.)
Coverity complains that r is overridden. In fact it isn't, but
we shouldn't set it like this anyway. exec_with_listen_fds() already
logs, so we only need to call _exit() if it fails.
CID#1646716
When the server side of a varlink connection supports connection
upgrades we need to go into single byte-read mode to avoid the
risk of a client that sends the json to protocol upgrade and then
immediately the custom protocol payload. This commit implements
this.
The next step is using MSG_PEEK to avoid the single-byte overhead.
This commit tweaks the API of sd_varlink_call_and_upgrade and
sd_varlink_reply_and_upgrade to return two independent fds even
if the internal {input,output}_fd are the same (e.g. a socket).
This makes the external API easier as there is no longer the risk
of double close. The sd_varlink_call_and_upgrade() is not in a
released version of systemd yet so I presume it is okay to update
it still.
This also allowed some simplifications in varlinkctl.c now that
the handling is easier.
With the new protocol upgrade support in varlinkctl client we can
now do the equivalent for the server side. This commit adds a new
`serve` verb that will serve any command that speaks stdin/stdout
via varlink and its protocol upgrade feature. This is the
"inetd for varlink".
This is useful for various reasons:
1. Allows to e.g. provide a heavily sandboxed io.myorg.xz.Decompress
varlink endpoint, c.f. xz CVE-2024-3094)
2. Allow sftp over varlink which is quite useful with the
varlink-http-bridge (that has more flexible auth mechanism than
plain sftp).
3. Makes testing the varlinkctl client protocol upgrade simpler.
4. Because we can.
The description of --exec is moved to a separate footer. It requires
special formatting and doesn't fit in the autogenerated table of
verbs.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
The -E short option previously used fallthrough into the --more case;
since macro-generated case labels don't support fallthrough (with some
older compilers), the --more logic is now duplicated inline in the -E
handler.
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
We generally want to print an error message if table_print()
fails. Add a helper function for this and use it consistently.
This does one of the three things depending on the call site:
- a no-change reformatting of the code
- change from a custom message to the generic one
- addition of the error message where previously none was printed
In the third case, the actual use impact is very small, since the
table formatting is very unlikely to fail. But if it did, we would
often return an error without any message whatsoever, which we
never want to do.
Almost all callers of table_print() specify stdout or NULL (equivalent
to stdout) as the output stream. Simplify things by not requiring the
the stream to be specified.
In almost all cases, the printing of the table is surrounded by normal
printfs() that don't do explicit flushing and for which we don't check
the output stream status. Let's simplify most callers and skip this
step. The reason is not so much to avoid the extra step itself, but
instead to avoid the _handling_ of the potential failure. We generally
only want to print an error message for ENOMEM and other "internal"
errors, so strictly speaking we should filter out the errors from the
stream. By skipping the flush&check step we implicitly do this.
Instead of exiting in exec_with_listen_fds() just return an error
and do the actual _exit() in the caller. Much nicer this way.
Thanks for Lennart for suggesting this.
The previous code used strv_join() when it generated the log
message for `varlinkctl --exec`. However this can lead to
inaccurate logging so use `quote_command_line()` instead.
Thanks to Lennart for suggesting this.
Having support for `--exec` when using `--upgrade` is nice so this
commit adds it. It does it by extracting a shared helper called
`exec_with_listen_fds()` and then use that in the `verb_call()`
and `varlink_call_and_upgrade()` calls.
The varlink spec supports protocol upgrades and they are very
useful to e.g. transfer binary data directly via varlink. So
far varlinkctl/sd-varlink was not supporting this. This commit
adds support for it in varlinkctl by using the new code in
sd-varlink and the generalized socket-forward code.
Typically, the top-level JSON object has to be an object, in any json
document we parse, hence let's add a simple way to enforce that.
Make use of this in various places.
(Note, various other JSON parsers insist on this logic right from the
beginning, but I actually thinking making this insisting optional like
this patch does it is the cleaner approach)
We often have a pattern where the same verb function is used for
multiple actions. This leads to an antipattern where we figure out what
action needs to be taken from argv[0] multiple times: often once in
parse_argv() to figure out what options are allowed, then once again
implicitly in dispatch_verb(), and then again in the action verb itself.
Let's allow passing a parameter into the verb to simplify this.
This matches a pattern we have in conf-parser.h, where we have both
void *userdata (more global) and void *data (per-config item). Here,
I opted for uintptr_t userdata. It seems that most of the time we'll
want to just pass an enum value. This works OK with no casts. I also
tried a void* and union. In both cases, much more boilerplate is needed:
either a cast or a macro to help avoid compiler warnings. uintptr_t
seems generic enough to cover foreseeable usecases with no fuss.
This is a noop refactoring. See next commit for an example.
When sd_varlink_connect_url() gets an unknown URL we now
check if there is a `$LIBEXECDIR/varlink-bridges/$scheme`
binary and execute it (with the url as the first arguments).
This makes varlink more flexible as it provides a way to
dynamically add "bridges" in LIBEXECDIR/varlink-bridges/. This is
conceptually similar to the libvarlink `varlink --bridge` command
and allows to e.g. call varlink over http{,s} via e.g. the new
varlink-http-bridge.
With a running varlink-http-bridge [0] one can do:
```console
$ varlinkctl call http://localhost:8080/ws/sockets/io.systemd.Hostname \
io.systemd.Hostname.Describe {}
{
"Hostname" : "top",
...
```
Closes: https://github.com/systemd/systemd/issues/40640
[0] https://github.com/mvo5/varlink-http-bridge/pull/1
Although extremely unlikely, there is a race present in solely checking the
$LISTEN_PID environment variable, due to PID recycling. Fix that by introducing
$LISTEN_PIDFDID, which contains the 64-bit ID of a pidfd for the child process
that is not subject to recycling.
We use symbols provided by unistd.h without including it. E.g.
open(), close(), read(), write(), access(), symlink(), unlink(), rmdir(),
fsync(), syncfs(), lseek(), ftruncate(), fchown(), dup2(), pipe2(),
getuid(), getgid(), gettid(), getppid(), pipe2(), execv(), _exit(),
environ, STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO, F_OK, and their
friends and variants, so on.
Currently, unistd.h is indirectly included mainly in the following two paths:
- through missing_syscall.h, which is planned to covert to .c file.
- through signal.h -> bits/sigstksz.h, which is new since glibc-2.34.
Note, signal.h is included by sd-eevent.h. So, many source files
indirectly include unistd.h if newer glibc is used.
Currently, our baseline on glibc is 2.31. We need to support glibc older
than 2.34, but unfortunately, we do not have any CI environments with
such old glibc. CIFuzz uses glibc-2.31, but it builds only fuzzers, and
many files are even not compiled.
We're planning to do code generation based on the systemd varlink
APIs. To simplify this, let's expose the IDL parser, so we can use
it to do code generation instead of having to write our own IDL
parser.
This commit cleans up the includes for all the small tools across
the tree.
A few cases of returning EXIT_SUCCESS are replaced with returning
0 to avoid including <stdlib.h>.
Split out of #37344.
Let's move some logic from _DEFINE_MAIN_FUNCTION() and other places
in main-func.h into functions that we implement in main-func.c to
allow moving some included headers from the header to the .c file.
Now that the necessary functions from log.h have been moved to macro.h,
we can stop including log.h in macro.h. This requires modifying source
files all over the tree to include log.h instead.
This new switch makes it possible to process fds attached to a varlink
reply: if specified it will execute a command line specified by the
caller, will pass the response json as stdin, and any fds acquired as
fd3, fd4, …
We often used a pattern like if (!FLAGS_SET(flags, SD_JSON_FORMAT_OFF)),
which is rather verbose and also contains a double negative, which we try
to avoid. Add a little helper to avoid an explicit bit check.
This change clarifies an aditional thing: in some cases we treated
SD_JSON_FORMAT_OFF as a flag (flags & SD_JSON_FORMAT_OFF), while in other cases
we treated it as an independent enum value (flags == SD_JSON_FORMAT_OFF).
In the first form, flags like SD_JSON_FORMAT_SSE do _not_ turn the json
output on, while in the second form they do. Let's use the first form
everywhere.
No functional change intended.
Initially I wasn't sure if this helper should be made public or just internal,
but it seems such a common pattern that if we expose the flags, we might just
as well expose it too, to make life easier for any consumers.
Currently, when SD_JSON_FORMAT_OFF is set in verb_call, the json format
flags are set to SD_JSON_FORMAT_PRETTY_AUTO|SD_JSON_FORMAT_COLOR_AUTO,
rather than or'ing those flags in. This means that other flags that may
have been set, e.g. SD_JSON_FORMAT_SEQ when --more is set, will be
clobbered.
Fix this by masking SD_JSON_FORMAT_OFF out, and then or'ing the new
flags in.
Let's provide a mechanism to select the number of screen columns for
rebreaking comments in Varlink IDL connected to a TTY, by honouring the
$COLUMNS env var then too. Previously we'd only honour when connected to
a TTY, but it's also useful otherwise for rebreaking ridiculously long
comments, hence honour it in this case too.
Inspired by #34098 → let's make it easier for users to understand and
correct the mistakes they made: let's early refuse invalid
interface/method names.
It's time. sd-json was already done earlier in this cycle, let's now
make sd-varlink public too.
This is mostly just a search/replace job of epical proportions.
I left some functions internal (mostly IDL handling), and I turned some
static inline calls into regular calls.
This is generally useful, but in some cases particularly: when
implementing enumeration calls that use the "more" flag to return
multiple replies then for the first reply we need to return an error in
case the list of objects to enumerate is empty, usually so form of
"NoSuchXYZ" error. In many cases this shouldn't really be treated as
error, as an empty list probably more than not is as valid as a list
with one, two or more entries.
Lennart Poettering
Zbigniew Jędrzejewski-Szmek
Michael Vogt
Luca Boccassi
Daniel Foster
Jelle van der Waa
Lennart Poettering
Yu Watanabe
Daan De Meyer
Zbigniew Jędrzejewski-Szmek
Nick Rosbrook
Mike Yuan
Luca Boccassi