Like the command-level CommandDescription.argspec, a verb's argspec is now
a nulstr, so that a verb may declare multiple alternative usage lines. The
verbs help table renders one synopsis line per nulstr entry in the
synopsis cell.
All verb argument specifications must gain an explicit NUL suffix. This
is a lot of churn, but I think this needs to be done. For now, there'll
be only one user of the new functionality (i.e. a verb with more than
one argspec), but I think we'll eventually want to use this in other
places too.
dns_cache_lookup() computes each answer item's expiry ('until', also the
basis of the SD_RESOLVED_CLAMP_TTL TTL patch) as "30s from now" whenever
the caller did not pass SD_RESOLVED_NO_STALE — keyed on the lookup mode,
not on whether the entry being returned is actually stale. That misfires
in both directions:
* A fresh entry hit by a serve-stale retry (second transaction attempt
with StaleRetentionSec= set) has its TTL reported as 30s instead of
its real remaining TTL.
* A lookup that passes neither SD_RESOLVED_NO_STALE nor
SD_RESOLVED_CLAMP_TTL — as the mDNS browse path does for clients
that set no flags — leaves 'current' at 0, so every answer item's
expiry becomes the constant absolute time 30s after boot,
permanently in the past. The RFC 6762 section 5.2 cache maintenance
schedule that resolved-dns-browse-services.c derives from
item->until then degenerates to its already-expired fallback.
Key the rewrite on the entry's actual staleness instead: an entry whose
TTL has lapsed keeps reporting "30s from now", so clients don't hold on
to stale data for long, and a fresh entry reports its real expiry. For
lookups with SD_RESOLVED_NO_STALE set nothing changes: a stale entry
already makes the whole chain miss, so every item returned in that mode
is fresh.
This converts systemd-homed, -hostnamed, -importd, -localed, -logind,
-machined, -networkd, -portebled, -resolved, -sysupdated, -timedated,
and -timesyncd.
The service name and description move from service_parse_argv() arguments
into a COMMAND() description that each daemon now defines. The
service_parse_argv() macro captures the calling binary's verb section,
where that COMMAND is found, while the options are still defined and
parsed in libshared, in the new "service" option namespace that each
COMMAND references. (The named namespace makes sure that options defined
by other translation units in libshared can never leak into the
services' parsers.)
The supported features are declared via option group filtering: each
COMMAND lists "Options" plus "Bus introspection" and/or "Runtime scope".
Unsupported options are now rejected by the parser as unknown options
(and are not shown in --help), replacing the runtime "This service does
not support …" errors.
The help output changes slightly: the "Bus introspection" and "Runtime
scope" option groups now get their own headings, and some descriptions
were reworded to the imperative form used elsewhere.
This time, various commands with non-trivial requirements are handled.
So the series is shorter, but has a pattern of feature additions and
then those new features being used. The way that --help output is
formatted is changed a bit in various places. Those changes are
described in individual commits.
22 'static int help()' functions left after this.
We already have io.systemd.service.SetLogLevel. Add a getter so that
CLI tools with `log-level` subcommands can be more easily implemented
with varlink.
Bind this method to all services that already bind io.systemd.service.SetLogLevel.
Serving stale cache data on a retry is a resilience feature for unicast
DNS: when the configured server does not respond, an expired answer beats
no answer. The gate in dns_transaction_go() that switches the cache lookup
into serve-stale mode from the second attempt on only checks
StaleRetentionSec=, not the scope's protocol, so it applies to mDNS and
LLMNR scopes as well. There it is wrong twice over:
* With StaleRetentionSec= set, a query for a peer that has left the
network misses the cache on the first attempt (the record's TTL has
expired), gets no reply, and is then answered from the expired but
retained record on the retry — for as long as the retention window
lasts. On the link-local protocols a retry means that no peer
answered, not that some server is unreachable, and RFC 6762 treats
TTL expiry as a presence signal: a host whose records lapse is gone.
* Independently of whether anything stale is cached, a cache hit made
in serve-stale mode is reported with its TTL clamped to 30s (see
dns_cache_lookup()) and counted in the served-stale statistics. mDNS
transactions are retried at 1s, 2s, 4s, ... whenever nobody answers,
so a record announced or learnt from another transaction in between
is handed to the client with a 30s TTL instead of its real remaining
TTL, and logged as a "Serve Stale response" although it never was
stale.
Hence only enter serve-stale mode on unicast DNS scopes. This is the
lookup-side counterpart of the cache-side change that stops link-local
records from being retained past their TTL at all.
In practice I think this will ever only be used with ansi_highlight_red(),
but it's cleaner to add this generic extension rather than e.g. a boolean
highlight_footer_red.
The options is passed through via command_print_help_full(), not through
the COMMAND structure, which means that commands with 'help' verb are not
covered by this. The red footer highligting is used for legacy commands,
which don't have verbs, so I think this is OK. Making it generic and
covering 'help' verbs would require more refactoring, and I think it
doesn't make sense to add it until it's actually needed.
Almost all programs are simple and have only a single command. When
printing options, let's just use the first COMMAND block found by default.
This means we can drop the explicit program-name argument and reduce
copy&paste boilerplate.
StaleRetentionSec= is a resilience feature for unicast DNS: when the
configured servers are unreachable, answering from an expired cache
entry beats answering nothing at all. That rationale does not carry
over to the link-local protocols, where there is no configured server
that could be unreachable -- peers simply come and go, and a record's
TTL is the only thing bounding how long a departed peer is still
answered for.
For mDNS the mismatch goes further. RFC 6762 treats record expiry as a
presence signal -- a host or service whose records lapse is to be
considered gone (section 10.1 goodbye packets even force this early by
publishing TTL=0), and the section 5.2 cache maintenance scheme exists
precisely so that expiry happens on time. Retaining expired mDNS
records for StaleRetentionSec= contradicts that: vanished peers linger
in the cache for the whole retention window, are served to clients as
if still present, and DNS-SD browsers do not see their services
disappear when the records' TTL lapses.
Beyond lingering entries, retention also swallows goodbyes outright, as
noted in review of the goodbye re-arm fix [1]: dns_cache_prune() and
dns_cache_expiry_in_one_second() key on the retention-extended "until",
so with StaleRetentionSec= set a goodbye's TTL=1 rewrite lands a full
retention window out -- the one-second prune of RFC 6762 section 10.1
then drops nothing, nothing re-arms, and the withdrawn service stays
listed for the whole window.
LLMNR has no goodbye mechanism, so only the lingering half applies to
it, but it does apply: dns_transaction_cache_answer() passes the
retention through for LLMNR replies just like for unicast DNS, and the
stale-serving gate in the transaction code does not look at the
protocol, so a departed LLMNR peer keeps being answered from cache for
the whole window as well.
Zero the retention centrally in dns_cache_put() for everything but
unicast DNS, which covers both put paths: transaction replies and the
mDNS packets cached directly by on_mdns_packet(). Add tests pinning
both directions, including the goodbye rewrite of an existing mDNS
entry.
[1] https://github.com/systemd/systemd/pull/43259#discussion_r3809488076
Add varlink version of FlushCaches.
Add this to io.systemd.Resolve.Monitor, since that interface already
handles many cache and manager related things.
The 'changed' variable is either unused on error or unconditionally
overwritten on success, so coverity warns about it. Drop the unused
assignment.
CID#1685266
Follow-up for a572dd2f60
BrowseServices(ifindex=0) may observe the same service on multiple
links. Treat the link index as part of the discovered service
identity so add and remove events remain link-scoped.
Use the per-service ifindex for RFC 6762 maintenance queries.
Share the service identity predicate between add and remove handling,
and propagate RR comparison errors instead of reporting removals.
Extend coverage for scoped-browser ifindex fallback, goodbye TTL
handling, and monotonic expiry updates.
So far there's only one multi-line abstract. But as pointed out in the
review, it's silly to handle footer but not the abstract.
help_abstract() is not reused because it only works with one-line
abstracts. (It always prints an empty line to separate paragraphs.)
So the formatting is done anew in print_wrapped(). I think this is OK,
since it's all very simple. After everything has been converted to
COMMAND() and its helpers, help_abstract() and the sister functions will
become unused and can be dropped then.
The footer is now wrapped to min($COLUMNS, 80) when it is printed, so
CommandDescription footers do not need to be hard-wrapped in the source,
and make use of the available terminal width. Newlines embedded in the
text are preserved and each line is wrapped independently, so explicit
line breaks can still be used where needed, e.g. before the example
command line in varlinkctl.
The text is wrapped at max of 80 columns to avoid very wrong lines on
wide terminals. Such lines are hard to read.
A new helper is added for this. We already have format_strv_width in
format-table, but it requires a strv, so we'd need to conver the data,
and returns a formatted string, which we'd need to print ourselves. So
overall, it seems better to add a dedicated helper.
Before this series, many help() functions opened the pager. This got lost
in the translation. Starting the pager requires access to arg_pager_flags.
While we could pipe this through in case when command_print_help() is
called directly, but this hard in the case when verb_help_auto() or
VERB_COMMON_HELP_AUTO/VERB_COMMON_HELP_AUTO_HIDDEN are used. Let's instead
extend the (const static) CommandDescription structure with a pointer to
the arg_pager_flags variable. This restores the pager functionality in a
uniform fashion with a one-line change in the individual binaries.
Previously, the systemd-resolve compat alias was described in the same
CommandDescription structure and resolvectl. But they have different
option sets, so this doesn't really work. The alias is deprecated, so
describe it as such in the abstract, but give it a separate entry.
$ build/resolvectl --introspect-cli
{
"mediaType": "application/vnd.io.systemd.cli-introspection-0",
"commands": [
{
"names": [ "resolvconf" ],
"abstract": [
"Register DNS server and domain configuration with systemd-resolved."
],
...
},
{
"names": [ "resolvectl" ],
"abstract": [
"Send control commands to the network name resolution manager, or\nresolve domain names, IPv4 and IPv6 addresses, DNS records, and services."
],
...
},
{
"names": [ "systemd-resolve" ],
"abstract": [
"Send control commands to the network name resolution manager, or\nresolve domain names, IPv4 and IPv6 addresses, DNS records, and services.\n\nThis command is deprecated. Use resolvectl.1 instead."
],
...
}
]
}
The resolvconf compatibility parser ignored errors from
nameserver, domain, and search lines. If one of those lines was
malformed, resolvectl could still send the partially parsed
configuration to resolved and overwrite the previous link state.
Reproducer:
IF=enp1s0
TMP="$(mktemp -d)"
ln -s "$(command -v resolvectl)" "$TMP/resolvconf"
sudo resolvectl dns "$IF" 1.1.1.1
sudo resolvectl domain "$IF" keep.example
resolvectl domain "$IF"
printf 'nameserver 1.1.1.1\nsearch "unterminated\n' |
sudo "$TMP/resolvconf" -a "$IF" || true
resolvectl domain "$IF"
Before:
The malformed search line was ignored and the previous domain
was cleared:
Link 2 (enp1s0): keep.example
Link 2 (enp1s0):
After:
The malformed input fails before updating resolved, so
keep.example remains configured.
Follow-up for: 097f130731
--raw only affects ResolveRecord output. Without --type= or an
RFC4501 dns: URI, query takes the ResolveHostname/ResolveAddress
paths, where --raw was silently ignored.
Reproducer:
build/resolvectl --raw query localhost
Before:
The command exited successfully and printed regular text output,
such as:
localhost: 127.0.0.1
After:
The command fails before issuing a lookup:
--raw may only be combined with --type= or dns: URIs.
Follow-up for: 05ea7e87b6
The service verb documents the one- and two-argument forms as
SRV lookups. They should not request TXT records by default.
Only the three-argument DNS-SD form should request TXT metadata.
Reproducer:
Run resolvectl service _mysvc._tcp signed.test in the
TEST-75-RESOLVED environment, where signed.test contains:
_mysvc._tcp SRV 10 5 1234 myservice
_mysvc._tcp TXT "This is TXT for myservice"
myservice A 10.0.0.20
Before:
resolvectl service _mysvc._tcp signed.test printed TXT data
for the plain SRV form:
_mysvc._tcp/signed.test: myservice.signed.test:1234 [priority=10, weight=5]
10.0.0.20
fd00:dead:beef:cafe::17
This is TXT for myservice
After:
The plain SRV form suppresses TXT data. The DNS-SD form still
shows it:
resolvectl service '' _mysvc._tcp signed.test
Follow-up for: a7a4c60a25
SetLinkDomains() marks old link domains and then builds the replacement
list in place. The marked old domains still counted against
LINK_SEARCH_DOMAINS_MAX, so replacing a non-empty list with exactly the
maximum number of new domains failed while adding the last entry. The error
path then unlinked all domains, so a failed setter call also destroyed the
previous configuration.
Reproducer:
resolvectl domain dnsbug1 old.example
busctl call org.freedesktop.resolve1 /org/freedesktop/resolve1 \
org.freedesktop.resolve1.Manager SetLinkDomains 'ia(sb)' "$ifindex" 1024 "${args[@]}"
resolvectl domain dnsbug1
Before: the bus call failed with "Argument list too long" and the link's
domain list became empty.
Follow-up for: d2ec6608b9
The DNS delegate parser tries to filter duplicate DNS= entries, but the
duplicate branch returned from the whole parser. This made a duplicate
server truncate the rest of the same DNS= line, silently dropping later
valid servers.
Reproducer:
DNS=192.168.77.78 192.168.77.78 192.168.77.79
Before: only 192.168.77.78 appeared in the delegate server list.
Follow-up for: 7928c0e0a1
The tlsa command accepts an optional service family before the domain list.
When the only argument was "tcp", "udp", or "sctp", we consumed it as the
family and returned success without issuing any query.
Reproducer:
build/resolvectl tlsa tcp
Before: the command printed nothing and exited with status 0.
Follow-up for: ebbc70e511
SetLinkDomains() tracks whether the configured search domain list changed
so it can skip redundant persistence. The route_only flag is part of the
D-Bus API and is persisted as a ~ prefix, but flipping an existing domain
between search and route-only only updated in-memory state and left
changed false. The method then skipped link_save_user() and
manager_write_resolv_conf(), leaving stale state until restart.
Reproduced on Ubuntu with systemd 259:
sudo resolvectl domain enp1s0 route-only-repro.test
sudo resolvectl domain enp1s0 '~route-only-repro.test'
resolvectl domain enp1s0
grep '^DOMAINS=' /run/systemd/resolve/netif/$(cat /sys/class/net/enp1s0/ifindex)
grep -H route-only-repro.test /run/systemd/resolve/*.conf
Before, resolvectl showed ~route-only-repro.test, but the persisted state
file still showed DOMAINS=route-only-repro.test, and the generated
resolv.conf files still contained search route-only-repro.test.
After this change, the state file is updated to
DOMAINS=~route-only-repro.test and the generated resolv.conf files are
refreshed so the route-only domain is not emitted as a search domain.
Follow-up for: cbf23f3853
When --json is requested, resolution errors returned by resolved still went
through text logging. That made scripts parse stderr even though the
varlink error already carried structured fields such as rcode and EDE.
Add a small JSON error dumper for resolvectl query errors. It outputs the
queried name and varlink error id alongside the original varlink parameters
so DNSError and DNSSECValidationFailed expose rcode, extendedDNSErrorCode,
extendedDNSErrorMessage, queryString, and result without duplicating the
schema in resolvectl.
Keep non-JSON text output and silent warn_missing NXDOMAIN handling
unchanged. Document the JSON resolution error object in resolvectl(1),
including the distinction between resolved errors and local invocation or
connection errors.
Closes#42904
The openpgp and tlsa verbs rejected --json= even though they
already determine the RR type they query. Allow JSON output for
these shortcut commands.
Accept matching explicit --type= values as well, but reject
mismatched types so openpgp only queries OPENPGPKEY records and
tlsa only queries TLSA records.
Refuse unauthenticated key records in JSON format so scripts do
not accidentally consume cryptographic DNS data that would only
produce a warning in human-readable output.
Fixes#43245
Since 4c0d8d9673, most dlopen notes are
set at the beginning of the executables. Let's manage all dlopen notes
there, rather than setting them where dlopen is called.
Note that the only exceptions are the LIBBPF_NOTE for networkd and
nsresource. Since dlopen_bpf() is wrapped in an `#if` guard, the notes
are instead set within the corresponding functionality.
As a result, the DLOPEN_FOO() wrapper macros are no longer needed and
can be dropped completely.
dns_question_cname_redirect() returns no replacement when a question
already matches a CNAME target or a DNAME does not apply. If only one of
the UTF-8 and IDNA questions redirects, dns_query_cname_redirect() then
installs NULL for the unchanged side.
Keep a reference to the original question on whichever side does not
redirect, and update the existing asymmetric DNAME test to verify it.
Follow-up for 23b298bce7
## Problem
A `BrowseServices` subscription with `"ifindex":0` (browse all
interfaces) receives a continuous flap of `added`/`removed` events for a
service that is still present — within a second, and with no goodbye
packet involved.
## Root cause
For `ifindex==0`, `mdns_browser_revisit_cache()` looked up each mDNS
scope's cache separately and called `mdns_manage_services_answer()`
**once per scope**. That function derives `removed` events by diffing
the browser's *global* discovered-service list (all interfaces, filtered
only by owner family) against the single answer it's handed. So with ≥2
mDNS-relevant interfaces of the same family, a service present on
interface A isn't in interface B's answer and is spuriously removed
while B is reconciled, then re-added on the next revisit tick.
## Fix
Accumulate the pruned cache answers from every matching mDNS scope into
one combined `DnsAnswer` and reconcile **once**, so removals diff the
global list against the union across interfaces. Items are merged with
`dns_answer_add_full()` (not `dns_answer_extend()`, which defaults each
item's `until` to `USEC_INFINITY` and would skew the RFC 6762 §5.2
TTL-maintenance schedule). The single-interface (`ifindex>0`) path is
unchanged.
## Test
`TEST-89-RESOLVED-MDNS.sh` gains `testcase_browse_ifindex_zero_no_flap`:
it adds a service-less dummy mDNS link to guarantee ≥2 same-family
scopes (the flap precondition), browses `ifindex=0`, waits for
discovery, then asserts **zero** `removed` events while every publisher
stays up. The subscription uses `varlinkctl --timeout=infinity`, since
it sits idle after discovery and the default 45s idle timeout would
sever it (and the assertion) mid-observation.
## Testing status
Builds clean; `shellcheck -x` clean. First CI round: `TEST-89`
(including this testcase) passed on all mkosi platforms; the failing
jobs all traced to unrelated flakes/infra.
dns_answer_remove_by_rr() mutats the backing OrderedSet even when the answer
has multiple references. Callers such as trust-anchor revocation could thus
change snapshots held elsewhere in the resolver.
Clone a shared answer before removal.
Follow-up for 71aee23dba
dns_zone_link_item() inserts a new item into the by-key hashmap before
publishing it by name. If the second hashmap insertion fails, _cleanup_
frees the item while by-key retains a dangling key and value.
Follow-up for 623a4c97b9
dns_add_new_service() links a _cleanup_ service into the browser
before copying its record and registering its maintenance timer has fully
succeeded. A timer setup failure then frees the service while leaving the
list head pointing at it.
Follow-up for 8458b7fb91
When a BrowseServices subscription uses ifindex=0 ("all interfaces"),
mdns_browser_revisit_cache() looked up each mDNS scope's cache separately
and called mdns_manage_services_answer() once per scope. That function
derives "removed" events by diffing the browser's *global* service list
(all interfaces, filtered only by owner family) against the single answer
it is handed. So with two or more mDNS-relevant interfaces of the same
family, a service present on interface A is not contained in interface B's
answer and is therefore spuriously removed (emitting "removed") while B is
reconciled, then re-added (emitting "added") on the next revisit tick.
The result is a continuous added/removed/added/removed flap, within a
second, for a service that is still present and without any goodbye packet
involved -- as reported for the mDNS/DNS-SD browsing API.
Fix it by accumulating the pruned cache answers from every matching mDNS
scope into a single combined answer and reconciling exactly once, so the
removed pass diffs the global service list against the union across all
interfaces. Each scope's cache is still pruned before lookup, so genuinely
expired records still drop out of the union and real removals still fire.
The items are merged with dns_answer_add_full() rather than
dns_answer_extend(), so each item's ifindex, flags, rrsig and cache-expiry
"until" are preserved. dns_answer_extend()/dns_answer_merge() are not
expiry-aware and do not carry the source items' "until" through: it
defaults to USEC_INFINITY (the "no expiry" sentinel), which would skew the
RFC 6762 section 5.2 TTL maintenance schedule that
mdns_manage_services_answer() derives from item->until.
The single-interface (ifindex>0) path is unchanged.
Previously, when a library was dynamically loaded via a helper function
inside libsystemd-shared.so, the resulting dlopen ELF note was not
propagated to the invoking executable's ELF metadata.
This commit explicitly annotates each executable with the relevant
dlopen ELF notes for any optional dependencies it might potentially
load. This ensures that package managers and build systems can properly
discover runtime dependencies that are triggered indirectly.
This also switches all these notes to be defined via SD_ELF_NOTE_DLOPEN_ANCHORED().
Consequently, any notes added within unreachable or unused functions will be
automatically garbage-collected by the linker (--gc-sections) instead of
bloating the final binary.
E.g. unused p11-kit library dependency is now dropped from
systemd-repart.standalone binary.
Before:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
idn Support for internationalized domain names libidn2.so.0 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
p11-kit Support for PKCS11 hardware tokens libp11-kit.so.0 suggested
```
After:
```
$ systemd-analyze dlopen-metadata build/systemd-repart.standalone
FEATURE DESCRIPTION SONAME PRIORITY
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 recommended
blkid Support for block device identification libblkid.so.1 required
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 recommended
mount Support for mount enumeration libmount.so.1 required
fdisk Support for reading and writing partition tables libfdisk.so.1 required
blkid Support for block device identification libblkid.so.1 recommended
libcrypto Support for cryptographic operations libcrypto.so.4 libcrypto.so.3 suggested
cryptsetup Support for disk encryption, integrity, and authentication libcryptsetup.so.12 suggested
fdisk Support for reading and writing partition tables libfdisk.so.1 suggested
mount Support for mount enumeration libmount.so.1 recommended
selinux Support for SELinux libselinux.so.1 recommended
tpm Support for TPM libtss2-esys.so.0 suggested
tpm Support for TPM libtss2-rc.so.0 suggested
tpm Support for TPM libtss2-mu.so.0 suggested
tpm Support for TPM libtss2-tcti-device.so.0 suggested
```
Zbigniew Jędrzejewski-Szmek
felix h
Christian Glombek
Nick Rosbrook
Yu Watanabe
dongshengyuan
Luca Boccassi
Lennart Poettering