json_stream_format_queue() drains queued output items into the output buffer and
stages their fds in n_output_fds, relying on the downstream sendmsg() to deliver
bytes-and-ancillary atomically as one SCM_RIGHTS message. If the output buffer
already holds bytes (from a prior fast-path enqueue that hasn't been sent yet or
from a partial write), concatenating a new fd-bearing item's JSON into it means
the next sendmsg() ships the combined bytes with those fds attached — violating
the per-message fd boundary on transports where that boundary is load-bearing.
Bail out of the drain loop when we would cross that boundary, so the next
write() first sends the buffered bytes with no ancillary, then pulls the
fd-bearing item into a clean buffer and ships it on its own sendmsg.
This only produces an observable difference for SOCK_SEQPACKET / SOCK_DGRAM
consumers: on those transports each sendmsg() is its own datagram with its own
SCM_RIGHTS cmsg, so whether we concatenate matters. On AF_UNIX SOCK_STREAM
(today's sole consumer shape, used by sd-varlink and the QMP client) the kernel
absorbs a preceding non-scm skb forward into the next scm-bearing skb's recv,
so per-sendmsg separation is invisible to the receiver anyway — the guard is
cheap defensive sender hygiene there, not a behaviour change. It becomes load-
bearing the moment a SEQPACKET/DGRAM consumer wires JsonStream up.
The json-stream API previously exposed JsonStreamQueueItem and several
functions operating on it (json_stream_make_queue_item(),
json_stream_enqueue_item(), json_stream_queue_item_free(),
json_stream_queue_item_get_data()). These existed solely to support
sd-varlink's "defer-and-modify" pattern for streaming replies, where a
reply is held back so its "continues" field can be set before
transmission. This is a varlink protocol concern that should not leak
into the generic transport layer.
Similarly, the fd pushing API (json_stream_push_fd(),
json_stream_reset_pushed_fds()) and the pushed_fds state lived inside
JsonStream, even though fd-to-message association is a protocol-level
concern managed entirely by sd-varlink.
Rework the API so that:
- JsonStreamQueueItem and all its functions become static to
json-stream.c. The only output API is now json_stream_enqueue_full()
(accepting explicit fds) and the inline json_stream_enqueue() wrapper
for the common no-fds case.
- The pushed_fds state moves from JsonStream into sd_varlink, where
sd_varlink_push_fd() and sd_varlink_reset_fds() manage it directly.
- The deferred reply in sd-varlink changes from a JsonStreamQueueItem*
to a plain sd_json_variant* plus a separate previous_fds/n_previous_fds
pair, keeping the protocol-specific bookkeeping in sd-varlink where it
belongs.
- A new varlink_enqueue() helper wraps json_stream_enqueue_full() with
the varlink connection's pushed fds, transferring fd ownership to the
queue item on success.
qmp-client.c is fixed to use the new API as well.
Move json_stream_description(), json_stream_log(), and
json_stream_log_errno() from json-stream.c into json-stream.h so that
consumers like the QMP client can use the same description-prefixed
logging that json-stream itself uses internally.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Convert string fields to varlink enums in io.systemd.Unit
Following
https://github.com/systemd/systemd/pull/39391#discussion_r2489599449,
convert all configuration setting fields in the io.systemd.Unit varlink
interface from bare SD_VARLINK_STRING to proper enum types, adding type
safety to the IDL.
This converts ~30 fields across ExecContext, CGroupContext, and
UnitContext, adding 25 new varlink enum types.
Weak compatibility breakage (per
https://github.com/systemd/systemd/pull/40972#issuecomment-4222294318):
Varlink enum identifiers cannot contain - or +, so affected values are
underscorified on the wire. For example, "tty-force" becomes tty_force,
"kmsg+console" becomes kmsg_console.
The full list of affected values:
```
- ExecInputType: tty-force, tty-fail
- ExecOutputType: kmsg+console, journal+console
- ProtectHome: read-only
- CGroupController: bpf-firewall, bpf-devices, bpf-foreign, bpf-socket-bind, bpf-restrict-network-interfaces, bpf-bind-network-interface
- CollectMode: inactive-or-failed
- EmergencyAction: exit-force, reboot-force, reboot-immediate, poweroff-force, poweroff-immediate, soft-reboot, soft-reboot-force, kexec-force, halt-force, halt-immediate
- JobMode: replace-irreversibly, ignore-dependencies, ignore-requirements, restart-dependencies
```
When JSON_STREAM_INPUT_SENSITIVE is set before the first read,
input_buffer is NULL, input_buffer_size is 0, and input_buffer_index
is 0. The old condition '!INPUT_SENSITIVE && index == 0' would route
this case into the else branch which calls memcpy() with a NULL source
pointer, which is undefined behavior even when the length is zero, and
is caught by UBSan.
Fix by checking input_buffer_index == 0 first, then allowing the
GREEDY_REALLOC fast path also when input_buffer_size == 0, since
there is no sensitive data to protect from realloc() copying in that
case. The else branch is now only entered when there is actual data
to copy (input_buffer_size > 0), guaranteeing input_buffer is
non-NULL.
Follow-up for 6b1a59d594
Introduces JsonStream, a generic transport layer for JSON-line message
exchange over a pair of file descriptors. It owns the input/output
buffers, SCM_RIGHTS fd passing, the deferred output queue, the
read/write/parse step functions, sd-event integration (input/output/time
event sources), the idle timeout machinery, and peer credential caching,
but knows nothing about the specific JSON protocol on top — the consumer
drives its state machine via phase/dispatch callbacks supplied at
construction.
sd-varlink is reworked to delegate the entire transport layer to a
JsonStream owned by sd_varlink. The varlink struct drops every
transport-related field (input/output buffers and fds, output queue,
fd-passing state, ucred/pidfd cache, prefer_read/write fallback, idle
timeout, description, event sources) — all of that lives in JsonStream
now. What remains in sd_varlink is the varlink-protocol state machine
(state, n_pending, current/previous/sentinel, server linkage, peer
credentials accounting, exec_pidref, the varlink-specific quit and defer
sources) and a thin wrapper layer over the JsonStream API. The
should_disconnect / get_timeout / get_events / wait helpers all live in
JsonStream now and are driven by a JsonStreamPhase the consumer reports
via its phase callback.
Define proper varlink enum types for unit configuration settings that
are part of the user-facing API (values users/clients can select).
This replaces SD_VARLINK_STRING with SD_VARLINK_DEFINE_FIELD_BY_TYPE
for these fields, giving them strong type semantics in the IDL.
Enum types added for ExecContext (ExecInputType, ExecOutputType,
ExecUtmpMode, ExecPreserveMode, ExecKeyringMode, MemoryTHP,
ProtectProc, ProcSubset, ProtectSystem, ProtectHome, PrivateTmp,
PrivateUsers, ProtectHostname, ProtectControlGroups, PrivatePIDs,
PrivateBPF), CGroupContext (CGroupDevicePolicy, ManagedOOMMode,
ManagedOOMPreference, CGroupPressureWatch, NFTSetSource, NFProto,
BPFCGroupAttachType, CGroupController), and UnitContext (CollectMode,
EmergencyAction, JobMode).
Engine-reported runtime state fields (Type, LoadState, ActiveState,
FreezerState, SubState, UnitFileState) remain as strings since only
the engine selects those values.
sd_json_variant_unsigned() incorrectly calls sd_json_variant_integer()
for reference-type variants instead of recursing to itself. This silently
returns 0 for unsigned values in the range INT64_MAX+1 through
UINT64_MAX, since sd_json_variant_integer() cannot represent them.
The sibling functions sd_json_variant_integer() and
sd_json_variant_real() correctly recurse to themselves.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Same pattern as the fix for sd_json_variant_unset_field in
9b3715d529: cache the element
count in a local variable and assert it is at least 2 before
subtracting.
CID#1548029
Follow-up for f2ff34ff2a
Typically, the top-level JSON object has to be an object, in any json
document we parse, hence let's add a simple way to enforce that.
Make use of this in various places.
(Note, various other JSON parsers insist on this logic right from the
beginning, but I actually thinking making this insisting optional like
this patch does it is the cleaner approach)
XDG_SESSION_EXTRA_DEVICE_ACCESS will now take a colon-separated list of
identifiers. For every identifier $ID, the session is granted access to all
devices tagged as "xaccess-$ID" in udev.
Fixes: #40634
For some reason the VS Code parser thinks sd_json_variant is still 48 bytes,
so skip the assert when running in that parser too
Follow-up for 337712e777
The assert at the bottom was updated when the size changed, but
the comment wasn't:
assert_cc(sizeof(sd_json_variant) == 40U);
Follow-up for 337712e777
name_to_handle_at_try_unique_mntid_fid() in its current form is
ill-designed for various reasons:
* AT_HANDLE_FID requires file system support, while unique mount id
is a VFS concept hence is always available if supported. Hence
the fallback for AT_HANDLE_MNT_ID_UNIQUE should be independent
of fid.
* The request for AT_HANDLE_MNT_ID_UNIQUE can be identified via
specifying ret_unique_mnt_id, no need for opening up the control
to caller (and currently the function simply doesn't handle
mismatch between ret params and flags).
* The caller cannot realistically differentiate whether the returned
mount id is actually unique.
* The path_get_unique_mnt_id() fallback did not handle AT_SYMLINK_FOLLOW.
Let's instead move the statx() fallback into name_to_handle_at_loop()
directly, and revamp interaction of ret_mnt_id/ret_unique_mnt_id:
if both are set, it indicates that the caller can handle both, hence
set what we have and return 0/1 for whether we managed to acquire
the unique one.
The !ret_handle && ret_mnt_id logic is removed. Let's not rely on
undocumented bizaare behavior and it's unused anyways.
path_get_mnt_id_at() exists for a reason...
Add JSON_BUILD_PAIR_IN_ADDR_WITH_STRING, JSON_BUILD_PAIR_IN6_ADDR_WITH_STRING,
and JSON_BUILD_PAIR_IN4_ADDR_WITH_STRING macros (plus _NON_NULL variants) that
emit both the binary address array and a human-readable string representation.
For example, an address field now emits both:
"Address": [192, 168, 1, 1]
"AddressString": "192.168.1.1"
This improves usability of the JSON/Varlink output for debugging and tooling
that consumes networkd state.
Updated networkd-json.c to use these new macros for addresses, neighbors,
nexthops, routes, routing policy rules, DNS servers, NTP servers, SIP servers,
domains, DHCPv6 prefixes, and link-local addresses.
Also updated the Varlink interface definitions to include the new string fields.
Let's move this from user-record.c into generic code and let's beef it
up a bit: allow parsing octal strings. This is particular relevant given
json_variant_new_stat() generates the mode in that format, and we
probably should be able to parse our own fields (even though we
currently don't do that for the data from json_variant_new_stat()).
In many cases we want to expose enums for which we have the usual
xyz_to_string()/xyz_from_string() via Varlink as enums. Let's add some
infra to test the tables against each other, to automatically detect
when they deviate.
In order to implement this properly, let's export/introduce clean
json_underscorefy()/json_dashify(), for dealing with the fact that our
enums usually use dash separates ames, but Varlink doesn't allow that.
(This does not add the test cases for all enum types we expose right
now, but only adds the general infra).
Let's use the proper uint32_t parsers initially, so that the usual logic
of formatting integers as decimal strings, works too for uids/gids. Not
because it made any sense to encode them like that, but just to be
systematic here.
Most of our dispatch helpers already do something useful in case they
are invoked on a null JSON value: they translate this to the appropriate
niche value for the type, if there is one.
Add the same for *all* dispatchers we have, to make this fully
systematic.
For various types it's not always clear which niche value to pick. I
opted for UINT{8,16,32,64}_MAX for the various unsigned integers, which
maps our own use in most cases. I opted for -1 for the various signed
integer types. For arrays/blobs of stuff I opted for the empty
array/blob, and for booleans I opted for false.
Of course, in various cases this is not going to be the right niche
value, but that's entirely fine, after all before a json value reaches a
dispatcher function it must pass one of two type checks first:
1. Either the .type field of sd_json_dispatch_field must be
_SD_JSON_VARIANT_TYPE_INVALID to not do a type check at all
2. Or the .type field is set, but then the SD_JSON_NULLABLE flag must be
set in .flags.
This means, accidentally generating the niche values on null is not
really likely.
Needed to implement support for RootHashSignature=/RootVerity=/RootHash=
and friends when going through mountfsd, for example with user units,
so that system and user units provide the same features at the same
level
When varlink parameters are empty, omit the "parameters" field entirely
rather than sending "parameters":{}. This reduces message size and
follows varlink specification which allows parameters to be omitted.
The implementation supports three equivalent representations for empty
parameters: field omission, JSON null, and empty object {}. All three
are accepted on input for backward compatibility.
Fixes: #38474
Let's not leak details from src/shared and src/libsystemd into
src/basic, even though you can't actually do anything useful with
just forward declarations from src/shared.
The sd-forward.h header is put in src/libsystemd/sd-common as we
don't have a directory for shared internal headers for libsystemd
yet.
Let's also rename forward.h to basic-forward.h to keep things
self-explanatory.
Daan De Meyer
Lennart Poettering
Christian Brauner
Luca Boccassi
Frantisek Sumsal
Ivan Kruglov
Yu Watanabe
Mike Yuan
Alessandro Astone
gvenugo3
Lennart Poettering
Daan De Meyer
Jelle van der Waa
Luca Boccassi