This substantially beefs up the local DNS stub feature set in order to
allow local clients to do DNSSEC validation through the stub.
Previously we'd return NOTIMP if we'd get a DO or DO+CD lookup. With
this change we'll instead:
1. If we get DO+CD requests (i.e. DNSSEC with no local checking) we'll
proxy DNS queries and response mostly unmodified to/from upstream DNS
servers if possible (this is called "bypass" mode). We will patch in
new request IDs, (and patch them back out on reply), so that we can
sanely keep track of things. We'll also maintain a minimal local
cache for such lookups, always keeping the whole DNS packets in it
(if we reply from cache we'll patch the TTLs of all included RRs).
2. If we get DO requests without CD (i.e. DNSSEC with local checking)
we'll resolve and validate locally. In this mode we will not proxy
packets, but generate our own. We will however cache the combination
of answer RRs (along with their packet section assignments) we got
back in the cache, and use this information to generate reply packets
from the DNS stub.
In both cases: if we determine a lookup is to be answered from LLMNR or
mDNS we'll always revert to non-DNSSEC, non-proxy operation as before.
Answers will lack the DO bit then, since the data cannot be validated
via DNSSEC by the clients.
To make this logic more debuggable, this also adds query flags for
turning off RR sources. i.e. cache/network/zone/trust anchor/local
synthesis may now be disabled individually for each lookup.
The cache is substantially updated to make all this work: in addition to
caching simple RRs for lookup RR keys, we'll now cache the whole packets
and the whole combination of RRs, so that we can answer DO and DO+CD
replies sensibly according to the rules described above. This sounds
wasteful, but given that the
DnsResourceRecord/DnsResourceKey/DnsAnswer/DnsPacket
objects are all ref-counted and we try to merge references the actual
additional memory used should be limited (but this might be something to
optimize further later on).
To implement classic RR key lookups and new-style packet proxy lookups
(i.e. the ones necessary for DO+CD packet proxying, as described above)
DnsTransaction and DnsQuery objects now always maintain either a
DnsResourceKey/DnsQuestion as lookup key or a DnsPacket for "bypass"
mode.
Fixes: #4621#17218
This is useful later when handling the truncation case: if we can't add
all RRs we'd like to add we need to set the TC bit, but still report the
number of RRs we added (and not the number of RRs we would have liked to
add) to the packet.
Let's beef up our parser a bit: let's store in the DnsAnswerFlags field
(that is stored as part of DnsAnswerItem) which DNS packet section (i.e.
answer, authoritative, additional) an RR originates from.
This is useful when propagating answers from an upstream DNS server
eventually, as we can place the data in the right sections downstream
too.
The more fields DnsAnswerItem gains the less sense it makes to pass
every field of it as separate parameter to an iterator macro. Let's
simplify things here, in preparation of adding more fields to the
structure later on: let's just return the structure itself in the loop,
rather than the individual fields.
This is a simple search&replace excercise: instead of accessing the
"key" field of the transaction directly, let's use a small inline helper
that does this for us.
This appears pointless for now, but this will become useful later when
we introduce "bypass" transactions, that reuse the original client DNS
packet for queries instead of synthesizing a packet of our own. In that
case transactions either have regular "key" field initialized as before,
or the "bypass" packet field instead. The new dns_transaction_key()
helper allows us to hide the differences for most cases as we can later
teach it to access the "bypass" packet's question key transparently.
No change in behaviour.
If we don't have ifindex info, don't set the field for it.
We already do that for parsed IP address replies, let's do it for all
cases: it's a bit nicer to suppress the ifindex prop if it doesn't apply
than to pass it invalid.
This is the other side of #18482, i.e. fixes things so that the parser
doesn't get tripped up by this.
(This too makes a problem go away we should track down properly, i.e.
figure out how the ifindex got lost in
https://github.com/systemd/systemd/pull/17823#issuecomment-742439422 )
I think this formatting was originally used because it simplified
adding new options to the help messages. However, these days, most
tools their help message end with "\nSee the %s for details.\n" so
the final line almost never has to be edited which eliminates the
benefit of the custom formatting used for printf() help messages.
Let's make things more consistent and use the same formatting for
printf() help messages that we use everywhere else.
Prompted by https://github.com/systemd/systemd/pull/18355#discussion_r567241580
GetMulticastHosts() returns an array of hostnames/addresses discovered via
LLMNR or Multicast DNS. It does not trigger any discovery on its own.
Instead, it simply returns whatever is already in resolved's cache.
Follow-up for 4ea8b443de. The logic that manages DnsQueryCandidate is rather
complicated: a calls to dns_query_complete() that wants to destroy a
DnsQueryCandidate can be nested inside a deep chain of calls. Using reference
counts seems like the simplest approach.
DnsSearchDomain already uses reference counting.
This patch effectively brings dns_query_candidate_go() to the state before
4ea8b443de, but wraps the iteration over DnsQueryCandidate.transactions in
dns_query_candidate_ref+dns_query_candidate_unref.
Even though many of those scripts are very simple, it is easier to include
the header than to try to say whether each of those files is trivial enough
not to require one.
@keszybz's right on
https://github.com/systemd/systemd/pull/18248#issuecomment-760798473:
swapping out the userdata pointer of a live varlink connection is iffy.
Let's fix this by making the userdata inheritance from VarlinkServer
object to the Varlink connection object optional: we want it for most
cases, but not all, i.e. all those cases where the calls implemented as
varlink methods are stateless and can be answered synchronously. For the
other cases (i.e. where we want per-connection objects that wrap the
asynchronous operation as it goes on) let's not do such inheritance but
initialize the userdata pointer only once we have it. THis means the
original manager object must be manually retrieved from the
VarlinkServer object, which in turn needs to be requested from the
Varlink connection object.
The userdata inheritance is now controlled by the
VARLINK_INHERIT_USERDATA flag passed at VarlinkServer construction.
Alternative-to: #18248
When dns_transaction_complete() manages to resolve a query, it invalidates the
query candidate object. It shall not be accessed afterwards.
We have the following chain of calls:
dns_query_candidate_go → dns_transaction_go → dns_transaction_prepare → dns_cache_lookup (success: 1)
→ dns_transaction_complete
After returning back to dns_query_candidate_go(), we'd attempt to continue
iteration over the list of transactions attached to the query candidate,
accessing already freed (and overwritten) memory:
(gdb) bt
0 0x00007f637297cf47 in hashmap_iterate_entry (i=0x7ffe7e15cc90, h=0x706f746b73656465) at ../src/basic/hashmap.c:703
1 _hashmap_iterate (h=0x706f746b73656465, i=i@entry=0x7ffe7e15cc90, value=value@entry=0x7ffe7e15cc88,
key=key@entry=0x0) at ../src/basic/hashmap.c:712
2 0x00007f637297d01b in set_iterate (s=<optimized out>, i=i@entry=0x7ffe7e15cc90, value=value@entry=0x7ffe7e15cc88)
at ../src/basic/hashmap.c:733
hence we crash
3 0x0000557bc99eb80f in dns_query_candidate_go (c=c@entry=0x557bcaf86890) at ../src/resolve/resolved-dns-query.c:139
...but c is not valid here in the second iteration of the loop
4 0x0000557bc99eb720 in dns_query_candidate_notify (c=0x557bcaf86890) at ../src/resolve/resolved-dns-query.c:271
c was valid here at entry...
5 0x0000557bc99efe28 in dns_transaction_complete (t=0x557bcac072f0, state=<optimized out>)
at ../src/resolve/resolved-dns-transaction.c:350
t is a valid transaction (11481 in the backtrace below)
6 0x0000557bc99f1efb in dns_transaction_process_reply (t=0x557bcac072f0, p=<optimized out>)
at ../src/resolve/resolved-dns-transaction.c:1171
7 0x0000557bc99f2d41 in on_dns_packet (s=<optimized out>, fd=<optimized out>, revents=<optimized out>,
userdata=0x557bcac072f0) at ../src/resolve/resolved-dns-transaction.c:1223
8 0x00007f6372a25217 in source_dispatch (s=s@entry=0x557bcb162c50) at ../src/libsystemd/sd-event/sd-event.c:3181
9 0x00007f6372a254fd in sd_event_dispatch (e=0x557bcb15b050) at ../src/libsystemd/sd-event/sd-event.c:3620
10 0x00007f6372a267c8 in sd_event_run (e=e@entry=0x557bcb15b050, timeout=timeout@entry=18446744073709551615)
at ../src/libsystemd/sd-event/sd-event.c:3678
11 0x00007f6372a269ef in sd_event_loop (e=0x557bcb15b050) at ../src/libsystemd/sd-event/sd-event.c:3700
12 0x0000557bc99ddc14 in run (argc=<optimized out>, argv=<optimized out>) at ../src/resolve/resolved.c:92
13 0x0000557bc99d260a in main (argc=<optimized out>, argv=<optimized out>) at ../src/resolve/resolved.c:99
xxx.name.net systemd-resolved[31705]: Got message type=method_call sender=:1.3644 destination=org.freedesktop.resolve1 path=/org/freedesktop/resolve1 interface=org.freedesktop.resolve1.Manager member=ResolveHostname cookie=2 reply_cookie=0 signature=isit error-name=n/a error-message=n/a
xxx.name.net systemd-resolved[31705]: idn2_lookup_u8: xxx → xxx
xxx.name.net systemd-resolved[31705]: Looking up RR for xxx IN A.
xxx.name.net systemd-resolved[31705]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=AddMatch cookie=1102 reply_cookie=0 signature=s error-name=n/a error-message=n/a
xxx.name.net systemd-resolved[31705]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=GetNameOwner cookie=1103 reply_cookie=0 signature=s error-name=n/a error-message=n/a
xxx.name.net systemd-resolved[31705]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.3324 path=n/a interface=n/a member=n/a cookie=4294967295 reply_cookie=1103 signature=s error-name=n/a error-message=n/a
xxx.name.net systemd-resolved[31705]: Cache miss for xxx.name.net IN A
xxx.name.net systemd-resolved[31705]: Transaction 11481 for <xxx.name.net IN A> scope dns on enp42s0/*.
xxx.name.net systemd-resolved[31705]: Using feature level UDP for transaction 11481.
xxx.name.net systemd-resolved[31705]: Using DNS server 192.168.1.1 for transaction 11481.
xxx.name.net systemd-resolved[31705]: Sending query packet with id 11481 of size 35.
xxx.name.net systemd-resolved[31705]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.3324 path=n/a interface=n/a member=n/a cookie=4294967295 reply_cookie=1102 signature= error-name=n/a error-message=n/a
xxx.name.net systemd-resolved[31705]: Match type='signal',sender='org.freedesktop.DBus',path='/org/freedesktop/DBus',interface='org.freedesktop.DBus',member='NameOwnerChanged',arg0=':1.3644' successfully installed.
xxx.name.net systemd-resolved[31705]: Processing incoming packet on transaction 11481 (rcode=NXDOMAIN).
xxx.name.net systemd-resolved[31705]: Not caching negative entry without a SOA record: xxx.name.net IN A
xxx.name.net systemd-resolved[31705]: Transaction 11481 for <xxx.name.net IN A> on scope dns on enp42s0/* now complete with <rcode-failure> from network (unsigned).
xxx.name.net systemd-resolved[31705]: Positive cache hit for xxx.lan IN A
xxx.name.net systemd-resolved[31705]: Transaction 64364 for <xxx.lan IN A> on scope dns on enp42s0/* now complete with <success> from cache (unsigned).
xxx.name.net systemd-resolved[31705]: Sent message type=method_return sender=n/a destination=:1.3644 path=n/a interface=n/a member=n/a cookie=1104 reply_cookie=2 signature=a(iiay)st error-name=n/a error-message=n/a
xxx.name.net systemd-resolved[31705]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=RemoveMatch cookie=1105 reply_cookie=0 signature=s error-name=n/a error-message=n/a
xxx.name.net systemd-resolved[31705]: Freeing transaction 64364.
xxx.name.net systemd[1]: systemd-resolved.service: Main process exited, code=dumped, status=11/SEGV
xxx.name.net systemd[1]: systemd-resolved.service: Failed with result 'core-dump'.
Fixes#16168, https://bugzilla.redhat.com/show_bug.cgi?id=1895937.
By default, systemd installs various sample configuration files
containing commented-out defaults. Systems seeking to minimize the
number of files in /etc may wish to install directories and
configuration files that have semantic effects, but not install not
commented-out sample configuration files.
Turn install-sysconfdir into a multi-valued option, with a "no-samples"
value to skip installing sample-only configuration files.
This fixes a bug introduced by ae45e1a383.
The set DnsAnswer::set_items contains the reference to the array in
DnsAnswer. So, the set must be reconstructed when we realloc() the
object.
Fixes#18132.
Previously, dns_answer_add() was O(n^2).
With this change dns_packet_extract() becomes ~15 times faster for some
extremal case.
Before:
```
$ time ./fuzz-dns-packet ~/downloads/clusterfuzz-testcase-minimized-fuzz-dns-packet-5631106733047808
/home/watanabe/downloads/clusterfuzz-testcase-minimized-fuzz-dns-packet-5631106733047808... ok
real 0m15.453s
user 0m15.430s
sys 0m0.007s
```
After:
```
$ time ./fuzz-dns-packet ~/downloads/clusterfuzz-testcase-minimized-fuzz-dns-packet-5631106733047808
/home/watanabe/downloads/clusterfuzz-testcase-minimized-fuzz-dns-packet-5631106733047808... ok
real 0m0.831s
user 0m0.824s
sys 0m0.006s
```
Hopefully fixes oss-fuzz#19227.
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19227
Lennart Poettering
Zbigniew Jędrzejewski-Szmek
Yu Watanabe
Daan De Meyer
Susant Sahani
Josh Triplett