Even after switching the wait loop to a polling `journalctl --grep`, the
test still fails intermittently because the very first messages emitted by
the freshly-spawned systemd-networkd-wait-online process can carry stale
journald metadata. journald associates `_SYSTEMD_UNIT=` (and friends) with
each entry by reading `/proc/$pid/cgroup` of the originating PID; if those
messages are produced before journald notices the cgroup migration into the
new service, they get tagged with `_SYSTEMD_UNIT=init.scope`. The
`-u $unit` filter then fails to match them.
Capture a journal cursor before launching the unit, and grep using
`--after-cursor=` plus `SYSLOG_IDENTIFIER=systemd-networkd-wait-online`
instead of `-u $unit`. SYSLOG_IDENTIFIER is set by the program itself, so
it's not subject to the cgroup-discovery race. The cursor bounds the search
to entries produced by this invocation, so prior wait-online runs in
earlier testcases don't interfere.
Logs from the failing run showing the messages exist but are tagged with
the wrong unit:
[ 2570.948554] TEST-75-RESOLVED.sh[2178]: + unit=wait-online-dns-ede81407-b93b-459d-8e5d-69292b42d2ae.service
[ 2571.023162] TEST-75-RESOLVED.sh[2178]: + systemd-run -u wait-online-dns-ede81407-b93b-459d-8e5d-69292b42d2ae.service ...
[ 2571.049189] TEST-75-RESOLVED.sh[2178]: + timeout 30 bash -c 'until journalctl -b -u wait-online-dns-ede81407-b93b-459d-8e5d-69292b42d2ae.service --grep ...'
[ 2571.964986] systemd-networkd-wait-online[2190]: dns0: No DNS server is accessible.
[ 2601.051088] TEST-75-RESOLVED.sh[2178]: ++ cleanup
And for that 2571.964986 entry:
_SYSTEMD_CGROUP=/init.scope
_SYSTEMD_UNIT=init.scope
_EXE=/usr/lib/systemd/systemd-executor
_CMDLINE=/usr/lib/systemd/systemd-executor --deserialize 68 ...
SYSLOG_IDENTIFIER=systemd-networkd-wait-online
MESSAGE=dns0: No DNS server is accessible.
Follow-up for d4bc62713e
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
The test used `timeout 30 bash -c "journalctl -b -u $unit -f | grep -m1 ..."`
to wait for systemd-networkd-wait-online to log that no DNS server is
accessible. The expected message is actually emitted ~1s after the unit
starts, but `grep -m1` exiting doesn't tear down `journalctl -f`: journalctl
only notices the closed pipe on its next write, which may never happen for
an otherwise idle unit. The pipeline therefore hangs until the 30s timeout
fires, eventually causing the test to fail.
Replace the follow+pipe with a polling `journalctl --grep` loop, which
exits cleanly as soon as the message lands in the journal.
Logs from the failing run:
[ 2650.871441] systemd-networkd-wait-online[2190]: dns0: No DNS configuration yet
[ 2651.723180] systemd-networkd-wait-online[2190]: dns0: No DNS server is accessible.
[ 2680.909048] systemd-networkd-wait-online[2190]: json-stream: Got POLLHUP from socket.
[ 2680.909092] systemd-networkd-wait-online[2190]: DNS configuration monitor disconnected, reconnecting...
[ 2680.914368] systemd-networkd-wait-online[2190]: Failed to connect to io.systemd.Resolve.Monitor: Connection refused
[ 2681.966674] systemd-networkd-wait-online[2190]: dns0: No DNS server is accessible.
[ 2681.969527] systemd-networkd-wait-online[2190]: Failed to connect to io.systemd.Resolve.Monitor: Connection refused
[ 2682.077032] systemd[1]: Stopping wait-online-dns-0f9e4f6d-8b34-4cff-b2da-03612ca731e8.service - [systemd-run] /usr/lib/systemd/systemd-networkd-wait-online --timeout=0 --dns --interface=dns0...
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
This is an extension of the /etc/hosts concept, but can provide any kind
of RRs (well, actually, we only parse A/AAAA/PTR for now, but the
concept is open for more).
Fixes: #17791
This makes it possible to have DNS requests for certain domains routed
differently than normal requests, which is for instance useful when
using policy routing to route traffic over a VPN but DNS requests for
the VPN endpoint itself, should be routed differently.
It doesn't make much sense to configure a firewall mark at the level of
a network interface, but at the level of a DNS delegate it can be very
useful.
Partially reverts d6b3793704.
On Ubuntu 26.04 devel, TEST-75-RESOLVED often fails at this stage,
despite the matching query apparently being found:
+ monitor_check_rr '2025-12-18 16:02:54' 'follow10.so.close.signed.test IN CNAME follow11.yet.so.far.signed.test'
+ set +x
Dec 18 16:02:54 ns1.unsigned.test resolvectl[552]: ← A: follow10.so.close.signed.test IN CNAME follow11.yet.so.far.signed.test
+ monitor_check_rr '2025-12-18 16:02:54' 'follow11.yet.so.far.signed.test IN CNAME follow12.getting.hot.signed.test'
+ set +x
Dec 18 16:02:54 ns1.unsigned.test resolvectl[552]: ← A: follow11.yet.so.far.signed.test IN CNAME follow12.getting.hot.signed.test
+ monitor_check_rr '2025-12-18 16:02:54' 'follow12.getting.hot.signed.test IN CNAME follow13.almost.final.signed.test'
+ set +x
Dec 18 16:02:54 ns1.unsigned.test resolvectl[552]: ← A: follow12.getting.hot.signed.test IN CNAME follow13.almost.final.signed.test
+ monitor_check_rr '2025-12-18 16:02:54' 'follow13.almost.final.signed.test IN CNAME follow14.final.signed.test'
+ set +x
Dec 18 16:02:54 ns1.unsigned.test resolvectl[552]: ← A: follow13.almost.final.signed.test IN CNAME follow14.final.signed.test
Unsetting pipefail, as was done before d6b3793704, appears to fix the
problem.
When a TTY is attached to the test unit, grep -q will generate SIGPIPE
for the previous command in the pipeline which in combo with `pipefail`
will cause the command to fail with exit status 141 which will fail the
test.
Replace with >/dev/null to avoid this from happening.
See also https://www.gnu.org/software/grep/manual/html_node/Usage.html
> There is a related problem with Bash’s set -e -o pipefail. Since grep
> does not always read all its input, a command outputting to a pipe read
> by grep can fail when grep exits before reading all its input, and the
> command’s failure can cause Bash to exit.
Co-authored-by: Yu Watanabe <watanabe.yu+github@gmail.com>
The test cases are not related to systemd-resolved.
While moving the test cases, now userdbctl is used for obtaining UID/GID
for the dynamic user, as musl does not support nss module, hence getent
does not provide information about the dynamic user.
Do not set a timeout on the wait-online call, since there are timeout
calls later that will prevent the test from blocking forever. Increase
those timeout calls for slower CI runs.
The current test is flaky because it creates a new interface definition,
calls networkctl reload, and then calls resolvectl show-cache. If
resolved has not received the changes and setup the DNS scopes for the
interface, show-cache will be empty for that interface.
Part 2 of #39602
Fixes the following warning:
TEST-75-RESOLVED.sh[2251]: ++ restart_resolved
TEST-75-RESOLVED.sh[2251]: ++ systemctl stop systemd-resolved.service
TEST-75-RESOLVED.sh[2271]: Stopping 'systemd-resolved.service', but its triggering units are still active:
TEST-75-RESOLVED.sh[2271]: systemd-resolved-monitor.socket, systemd-resolved-varlink.socket
It looks like the 4 second sleep might not be enough on some slower
machines (like the ARM GH Actions nodes) which can lead to the DS RRs
propagation to clash with the manual test zone edit, and the
signed.test zone then might end up not properly signed:
TEST-75-RESOLVED.sh[749]: + : '--- ZONE: signed.test (static DNSSEC) ---'
TEST-75-RESOLVED.sh[749]: + run_delv @ns1.unsigned.test signed.test
TEST-75-RESOLVED.sh[749]: + run delv -a /etc/bind.keys @ns1.unsigned.test signed.test
TEST-75-RESOLVED.sh[778]: + delv -a /etc/bind.keys @ns1.unsigned.test signed.test
TEST-75-RESOLVED.sh[779]: + tee /tmp/tmp.2KOIiyrgth
TEST-75-RESOLVED.sh[779]: ;; /etc/bind.keys:1: option 'managed-keys' is deprecated
TEST-75-RESOLVED.sh[779]: ;; validating signed.test/DS: no valid signature found
TEST-75-RESOLVED.sh[779]: ;; validating signed.test/A: no valid signature found
TEST-75-RESOLVED.sh[779]: ; unsigned answer
TEST-75-RESOLVED.sh[779]: signed.test. 86400 IN A 10.0.0.10
TEST-75-RESOLVED.sh[779]: signed.test. 86400 IN RRSIG A 13 2 86400 20251028114356 20251014101356 39330 signed.test. oo3ca8WPusbBPRhzsEKw3bsBBqFtI8i4bckoMVNzt7lY+udGW6PlaSYj OjpQGgY9oglowVM9bteNtwJKHUbvtw==
TEST-75-RESOLVED.sh[749]: + grep -qF '; fully validated' /tmp/tmp.2KOIiyrgth
[FAILED] Failed to start TEST-75-RESOLVED.service - TEST-75-RESOLVED.
Let's explicitly wait for the DS records propagation to finish before we
start editing the test zone to avoid this.
I'm still not completely sure if this is the root cause, but it's the
best shot I currently have, so I'll let the CIs decide.
We passes log level through kernel command line. It is not necessary to
set to debug level at the beginning, and set to info at the end.
This is important when a test has several subtests. If a subtest sets
log level to info at the end, then subsequent tests may not generate any
useful logs.
This also makes shebang always use env command, and drops unnecessary
'bash -c' or 'sh -c' when a signle command is invoked in the shell,
like sleep or echo.
There are a lot of resolved.conf.d drop-ins used in these tests. Use
proper numeric prefixes, especially to avoid confusion with sorting
relative to test.conf.
Make the test base config 10-test.conf, and use 90-*.conf elsewhere.
Specifically, add a test case that ensures systemd-networkd-wait-online --dns
is robust against (a) systemd-resolved absence, and (b) systemd-resolved
restarts.
bind9 9.21 removed the deprecated 'managed-keys', swap it with
'trust-anchors' if the version is 9.21 or newer
[ 20.654086] TEST-75-RESOLVED.sh[1217]: + delv -a /etc/bind.keys @ns1.unsigned.test signed.test
[ 20.654425] TEST-75-RESOLVED.sh[1218]: + tee /tmp/tmp.D4LNomAKqY
[ 20.672599] TEST-75-RESOLVED.sh[1218]: ;; /etc/bind.keys:1: option 'managed-keys' no longer exists
Based on this patch i had submitted to RedHat
(https://issues.redhat.com/browse/RHEL-56280), i am submitting this
patch to this upstream systemd.
There is no way to explicitly enable/disable IPv6 AAAA queries.
Problem was that i am using RHEL9 and some applications does not use a
newer glibc that supports `no-aaaa` option in `/etc/resolv.conf`. So
some applications will still resolve IPv6 AAAA even with `no-aaaa`
option and it is inconsistent across the system where some work and some
don't.
So this systemd-resolved patch catch-all queries and disable IPv6 AAAA
queries for all applications in the OS by having an option
`RefuseRecordTypes=AAAA` to disable IPv6 AAAA queries.
Although https://github.com/systemd/systemd/pull/28136 tries to fix this
automatically but it still does not work with
`net.ipv6.conf.all.disable_ipv6 = 1`. Also tried with explicitly
removing the conditional and force set `family = AF_INET` and still
resolves AAAA records.
The issue is that i want to explicitly disable IPv6 AAAA queries instead
of systemd-resolved to figure out itself which address family it is
using, which always have problems.
CNAME doesn't exist at the zone apex. When we get an unsigned noerror
response to a direct query for a CNAME record, we don't yet know if this
name is zone apex. We already request the correct DS record in this
case, but previously skipped it at validation time, causing the answer
to appear bogus. Make sure to also consider the DS record for the query
name for negative replies.
Although being far from ideal and the first two test cases have to be run
before the setup phase otherwise they will fail, it still makes the test
suite look much better and easier to read
Having these named differently than the test itself mostly creates
unecessary confusion and makes writing logic against the tests harder
so let's rename the testsuite-xx units and scripts to just use the
test name itself.
Luca Boccassi
Lennart Poettering
r-vdp
Nick Rosbrook
Daan De MeyerandYu Watanabe
Zbigniew Jędrzejewski-Szmek
Frantisek Sumsal
Lennart Poettering
Muhammad Nuzaihan Bin Kamal Luddin
Muhammad Nuzaihan Bin Kamal Luddin
Ronan Pigott
Daniel Rusek