This is a followup to https://github.com/systemd/systemd/pull/41400 with
the final set of tweaks so that the new coccinelle `check-pinters-deref`
checker runs without failures.
It also includes a commit with some fixes for redundant asserts and the
final commit to make the assert() in the inner loop of qsort
POINTER_MAY_BE_NULL which will become a no-op - that is probably the
most controversial one, I hope the commit message explains the
trade-offs, I'm happy to drop it, I have no strong opinion either way.
Please see the individual commits/commit messages for more details, most
is (hopefully) relatively boring/mechanical.
This commit fixes an issue with the fd handling in
sd_varlink_call_and_upgrade() when one direction of the output
FDs is unset.
Thanks to Lennart for spotting this and suggesting the fix.
The varlink spec supports protocol upgrades and they are very
useful to e.g. transfer binary data directly via varlink. So
far sd-varlink was not supporting this.
This commit adds a new public sd_varlink_call_and_upgrade()
that sends a method call, waits for the reply, then steals
the connection fds for raw I/O. It returns separate input_fd
and output_fd to support both bidirectional sockets and pipe
pairs.
A helper is extracted and shared between sd_varlink_call_full()
and sd_varlink_call_and_upgrade(). A new `protocol_upgrade`
bool in `struct sd_varlink` ensures that on a protocol upgrade
request we only exactly read the varlink protocol bytes and
leave anything beyond that to the caller that speaks the upgraded
protocol.
Note that this is the client side of the library implementation
only for now. The server side needs work but this is already
useful as it allows to talk to varlink servers that speak protocol
upgrades (like the rust implemenations of varlink).
Use CONST_ALIGN_TO to express the compile-time overflow check for
the ALIGN(sizeof(sd_bus_message)) + sizeof(BusMessageHeader)
allocation, since ALIGN() is not constexpr.
CID#1548031
Follow-up for de1c301ed1
Coverity flags arithmetic in BUS_MESSAGE_SIZE(),
BUS_MESSAGE_BODY_BEGIN() and message_from_header() as potential
overflows. The values are validated at message creation time, but
add asserts to make the invariants explicit for static analyzers.
CID#1548023
CID#1548030
CID#1548046
Follow-up for 6629161f82
Coverity flags the subtraction from msg_controllen as a potential
underflow. The CMSG_SPACE was added when send_ucred was set, and
the subtraction only runs when send_ucred was true, so it is safe.
Add an assert to document this invariant.
CID#1548074
Follow-up for 64144440a5
Same pattern as the fix for sd_json_variant_unset_field in
9b3715d529: cache the element
count in a local variable and assert it is at least 2 before
subtracting.
CID#1548029
Follow-up for f2ff34ff2a
Coverity flags max_size*2 as a potential overflow. The value is
bounded by MAX_SIZE_UPPER (128 MiB) or JOURNAL_COMPACT_SIZE_MAX
(4 GiB), so doubling is safe within uint64_t. Add an assert to
document this.
CID#1548019
Follow-up for 8580d1f73d
I tried to implement a varlink service using sd-varlink, and
not being able to use the approach with sentinel is exteremely
painful. This is useful internally and likewise externally.
Coverity flags rbuffer_size += k as a potential overflow, but k is
always bounded by the iov size (which is the difference between the
allocated buffer and current rbuffer_size). Add asserts to make this
invariant explicit.
CID#1548044
CID#1548071
Follow-up for a7e3212d89
Coverity flags the signal_sources array access as a potential
out-of-bounds read because it cannot trace through the SIGNAL_VALID()
macro to know that ssi_signo < _NSIG. Add an explicit assert after
the runtime check to make the constraint visible to static analyzers.
CID#1548033
Follow-up for 7a64c5f23e
External tools that use the systemd varlink ecosystem require
to know if a specific varlink method supports/requires the
"more" flag from the IDL. This is tracked upstream in
https://github.com/varlink/varlink.github.io/issues/26
As an intermediate step systemd adds the (very nice) comments
```
# [Requires 'more' flag]
or
# [Supports 'more' flag]
```
to the various methods.
This commit extends the comment around the code that adds the
comment to clarify that this should be considered API and that
the comment should not be changed as external tools (like e.g.
the varlink-http-bridge) rely on it.
This is an another alternative for #40656, based on @poettering's
suggestion to use an ELF section.
The output of --help is generated using format-table, but the details of
the formatting a bit off: different sections in the table (verbs, option
groups) are not aligned to the same column. Our current table formatting
doesn't make this easy. If somebody has an idea how to do this without
too much pain, please make suggestions.
Another thing that I didn't know how to do nicely, was to use the two
columns of separation. But maybe this doesn't matter. If we switch to
one column everywhere, I don't think people will care.
Example output:
```console
$ build/systemd-id128 -h | cat
systemd-id128 [OPTIONS...] COMMAND
Generate and print 128-bit identifiers.
Commands:
new Generate a new ID
machine-id Print the ID of current machine
boot-id Print the ID of current boot
invocation-id Print the ID of current invocation
var-partition-uuid Print the UUID for the /var/ partition
show [NAME|UUID] Print one or more UUIDs
help Show this help
Options:
-h --help Show this help
--version Show package version
--no-pager Do not start a pager
--no-legend Do not show headers and footers
--json=FORMAT Output inspection data in JSON (takes one of pretty,
short, off)
-j Equivalent to --json=pretty (on TTY) or --json=short
(otherwise)
-p --pretty Generate samples of program code
-P --value Only print the value
-a --app-specific=ID Generate app-specific IDs
-u --uuid Output in UUID format
See the systemd-id128(1) man page for details.
```
The output wraps automatically with terminal width. If the terminal is
extremely narrow, ellipsization occurs. I think this doesn't matter
and/or is actually a feature. Such narrow terminals mostly occur in
testing, so it doesn't matter what exactly we do, as long as it is
something somewhat reasonable.
Anyway, I think this is enough as PoC. Please compare this with the
previous approach. /cc @behrmann, @YHNdnzj, @poettering
The three binaries that are converted have options, verbs, optional
arguments, and options terminate option parsing, so most of the
functionality is there. One thing that I didn't implement that was
present in previous PRs is "namespaces", i.e. multiple parsers in the
same source file. I expect that we can handle this similarly to option
groups.
In one of the reviews one of the LLMs noticed that the pragma is set but
never unset, so it remains in effect for the rest of the translation
unit. From the comment, it's not clear how old those "old compilers"
were, so let's try if things work without this workaround.
Typically, the top-level JSON object has to be an object, in any json
document we parse, hence let's add a simple way to enforce that.
Make use of this in various places.
(Note, various other JSON parsers insist on this logic right from the
beginning, but I actually thinking making this insisting optional like
this patch does it is the cleaner approach)
dbus-daemon intentionally disconnects peers when FDs get
truncated. Detect it and skip it in that case, as the purpose
of the test is not to exercise the D-Bus implementation, but
our library.
When running with dbus-broker (Fedora, etc) we'll get full
coverage.
Fixes https://github.com/systemd/systemd/issues/41150
Follow-up for 744d589632
In one of the reviews one of the LLMs noticed that the pragma is set but
never unset, so it remains in effect for the rest of the translation
unit. From the comment, it's not clear how old those "old compilers" were,
so let's try if things work without this workaround.
Properties are set through uevent, udev rules, or program output by IMPORT.
They may contain spurious characters and udev database parsers may be confused.
Let's refuse spurious properties.
The function device_add_property() handles property with NULL value as
removing the property. Let's also make an empty value is handled as
same.
Note, ENV{hoge}=="" udev property handles both NULL value and an empty string
in the same way.
Let's make sure the parameters/error pointers into the message remain
valid as long as the reply callback is running, even if the reply
otherwise resets the pending message.
Suppress warnings like the following from clang tidy:
```
../src/boot/addon.c:11:19: error: function 'efi_main' can be made static to enforce internal linkage [misc-use-internal-linkage,-warnings-as-errors]
11 | EFIAPI EFI_STATUS efi_main(EFI_HANDLE image, EFI_SYSTEM_TABLE *system_table);
| ^
```
Some warnings are suppressed simply by setting comments to ignore the warning,
some are by making global variables static, or include a suitable header.
test_sd_device_enumerator_filter_subsystem hangs forever in OBS builds,
as it waits for the udev queue to be drained. But this is not a booted
system, with systemd and udev, so nothing will do that.
Skip the test in that case.
The callback might drop the last ref to the slot object, and we still
want to access it. Hence do what we usually do in these cases: keep an
extra reference while processing the callback.
The removed code in device_tag_index() in fact does nothing,
as sd_device.all_tags is never cleared. Moreover, not only the code
is meaningless, but it is theoretically/logically wrong, as the symlinks
in /run/udev/tags/ should be 'sticky', hence we should even not try to
remove them.
The current tag concept has been introduced by
e77b146f82 (v247) to make symlinks in
/run/udev/tags/ are 'sticky'.
However, when TAG= (rather than TAG+=) is specified, then the tags
assigned in the previous events were also cleared.
This fixes the issue and now symlinks in /run/udev/tags/ are really
'sticky'.
Fortunately, TAG= is mostly unused. So, the issue should not affect
and the fix should not change anything on almost all systems.
Let's check if we're already in the right namespaces and call connect()
directly if that's the case. This can easily happen when the machine is
specified as .host or so.
Zbigniew Jędrzejewski-Szmek
Michael Vogt
Luca Boccassi
Daan De Meyer
Lennart Poettering
Frantisek Sumsal
seaeunlee
Yu Watanabe
Mike Yuan