Commit Graph
51373 Commits
Author SHA1 Message Date
Zbigniew Jędrzejewski-Szmek fa10451686 units: strip out the developer comment in .service unit again
The comment talks about upstream development steps and doesn't make
sense for users. We used special '## ' syntax to strip it out during
build, but it got inadvertently reformatted as a normal comment
in 3982becc92.
2021-05-19 10:24:43 +09:00
Zbigniew Jędrzejewski-Szmek 059cc610b7 meson: use jinja2 for unit templates
We don't need two (and half) templating systems anymore, yay!

I'm keeping the changes minimal, to make the diff manageable. Some enhancements
due to a better templating system might be possible in the future.

For handling of '## ' — see the next commit.
2021-05-19 10:24:43 +09:00
Zbigniew Jędrzejewski-Szmek 6b1aac3ccf meson: replace some m4 templates with jinja2
m4 was nice in '85, but the syntax feels a bit dated. Since we use python for
meson, let's use a popular python templating engine to replace some m4 usage.

A little nicety is that typos are caught:

FAILED: sysusers.d/systemd-remote.conf
/usr/bin/meson --internal exe --capture sysusers.d/systemd-remote.conf -- /home/zbyszek/src/systemd/tools/meson-render-jinja2.py config.h ../sysusers.d/systemd-remote.conf.j2
Traceback (most recent call last):
  File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 28, in <module>
    print(render(sys.argv[2], defines))
  File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 24, in render
    return template.render(defines)
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3.9/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 8, in top-level template code
jinja2.exceptions.UndefinedError: 'HAVE_MICROHTTP' is undefined

This checking mirrors what 349cc4a507 did for C defines.
2021-05-19 10:24:43 +09:00
Zbigniew Jędrzejewski-Szmek e0698c66f4 meson, mkosi: pull in jinja2 as required
In preparation for subsequent changes.
2021-05-19 10:24:43 +09:00
Zbigniew Jędrzejewski-Szmek be21b60b89 modules-load: inline interator variable 2021-05-19 10:24:43 +09:00
Zbigniew Jędrzejewski-Szmek 6b908313ef test/run-unit-tests: make script compatible with python3.6
dataclasses were added in python3.7, but bionic has python3.6.
Yes, the new code is a travesty, but it does the job.

Fixes #19640.
2021-05-18 23:59:05 +02:00
Yu Watanabe cc2d7efc5c network: fix Link reference counter issue
Previously, when link_new() fails, `link_unref()` was called, so,
`Manager::links` may become dirty.
This introduces `link_drop_or_unref()` and it will be called on
failure.
2021-05-18 20:40:56 +09:00
Yu Watanabe 56001f0233 network: also drop requests when link enters linger state
Otherwise, if link is removed, several references to the link in remain
exist in requests.
2021-05-18 20:40:56 +09:00
Yu Watanabe 63130eb36d network: merge link_drop() and link_detach_from_manager()
link_detach_from_manager() is only called by link_drop(). It is not
necessary to split such tiny function.
2021-05-18 20:40:56 +09:00
Yu Watanabe ce9388d72a network: fix possible message counter double decrement
When after_configure() for a request fails, then the request is not
removed from the queue at that time, and the link enters to failed
state. After that, if the link is reconfigured or its carrier is lost,
then the request is dropped from the queue, and the message_counter is
decreased. However, the counter is already or will be also decreased
when the corresponding netlink reply is received.
So, the counter is decreased twice.
2021-05-18 20:40:50 +09:00
Yu Watanabe 72ffb9133d test-network: add testcases for ServerAddress= 2021-05-18 20:20:24 +09:00
Yu Watanabe 0017ba3165 network: dhcp-server: introduce ServerAddress= setting
This may be useful when the link which DHCP server running on has
multiple static addresses.
2021-05-18 20:20:24 +09:00
Yu Watanabe 998545a7d9 network: address: introduce link_get_ipv4/ipv6_address() 2021-05-18 20:20:24 +09:00
Yu Watanabe 99b5f4f7b8 network: use UINT32_C() macro 2021-05-18 20:20:24 +09:00
Yu Watanabe 473680be32 network: introduce address_set_broadcast() 2021-05-18 20:20:20 +09:00
Yu Watanabe c9f2db2c40 network: use config_parse_in_addr_non_null() 2021-05-18 20:19:37 +09:00
Yu Watanabe cf0747725d conf-parser: introduce config_parse_in_addr_non_null() 2021-05-18 20:19:37 +09:00
Anita Zhang c54cfef396 man: only the system manager does not pass vars in ManagerEnvironment=
User managers always pass their environment on to their children.
Make that clear in the description of ManagerEnvironment= which
states that none of those args will get passed to child processes of
service managers.
2021-05-18 12:59:10 +02:00
Jörg Thalheim f96bc66901 fix bash shebangs 2021-05-17 22:27:41 +01:00
Olle Lundberg c920b86364 nspawn: bring back the word may in error text
In the change set 6c045a9998 the error
text for the old flag `--private-users-chown` was repurposed for the
new flag `--private-users-ownership=own` and while doing so the word
`may` was dropped leading to a grammatically incorrect error text.
2021-05-17 22:21:00 +01:00
Balint Reczey 39f1bdecc2 test: Allow running only a subset of integration tests by setting SELECTED_TESTS 2021-05-17 14:16:09 +01:00
Zbigniew Jędrzejewski-Szmek 9578b472f4 mkosi: stop pulling in workaround packages
libfido2 was pulled in recently [1], and the dependency on libzstd has been
present for a long time.

nano is now the default editor in Fedora [2], but we still want to pull it in
to have an editor in minimal images.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1949505
[2] https://fedoraproject.org/wiki/Changes/UseNanoByDefault
2021-05-17 11:14:03 +01:00
Frantisek Sumsal 35a48d4b9a ci: build with gcc-11 as well
Focal builds for gcc-11 are finally available, so let's put them into
some use.
2021-05-16 21:28:04 +01:00
ei-ke 410477a78f Fixed a typo 2021-05-16 18:28:58 +02:00
Milan 21dfadbde0 docs: Update link to journal file format codebase 2021-05-16 16:51:18 +01:00
Milan 882da5cca7 docs: Update link to journal-def.h 2021-05-16 16:51:18 +01:00
tramsay 7b5018ca9e network:dhcp4_server_configure returns if not able to get timezone
When /etc/localtime is a symbolic link pointing to another symbolic
link, get_timezone will return -EINVAL instead of the timezone.
This issue can cause systemd-networkd DHCPServer to fail.

Instead of returning failure, log a warning indicating that that
the timezone will not be sent.

	modified:   networkd-dhcp-server.c
2021-05-15 18:14:49 +02:00
Sebastian Blunt 1fa94a3124 Add crypttab option silent
Adds a crypttab option 'silent' that enables the AskPasswordFlag
ASK_PASSWORD_SILENT. This allows usage of systemd-cryptsetup to default
to silent mode, rather than requiring the user to press tab every time.
2021-05-15 16:23:56 +02:00
Dan Streetman d57e871c60 test: combine stdout/stderr from failed test
Printing stdout and stderr from a failed test makes it harder to
interpret what the specific problem was; instead let's print out
the lines in order as we got them when the test was run

Also save failed test output to file if ARTIFACT_DIRECTORY is defined
2021-05-15 11:49:24 +01:00
Zbigniew Jędrzejewski-Szmek f1e696221f Merge pull request #19612 from keszybz/meson-0.58-quieting
Make meson output quiet again
2021-05-15 10:49:10 +02:00
Zbigniew Jędrzejewski-Szmek c856ef0457 resolved: fix braino with reference counting and linked lists
In 0e0fd08fc8 I added reference counts to keep
track of the DnsQueryCandidate objects. Unfortunately, dns_query_unref_candidates()
was written as

     while (q->candidates)
           dns_query_candidate_unref(q->candidates);

i.e. it would keep dropping the reference count as many times as needed for it
to hit 0, making the patch less than fully effective.

dns_query_unref_candidates() is renamed to dns_query_detach_candidates() and
changed to drop exactly one reference from each of the linked candidates.

Example failure:
==463== Invalid read of size 8
==463==    at 0x419C93: dns_query_candidate_go (resolved-dns-query.c:159)
==463==    by 0x41A143: dns_query_candidate_notify (resolved-dns-query.c:304)
==463==    by 0x434BD6: dns_transaction_complete (resolved-dns-transaction.c:437)
==463==    by 0x436A0F: dns_transaction_process_dnssec (resolved-dns-transaction.c:976)
==463==    by 0x4378C1: dns_transaction_process_reply (resolved-dns-transaction.c:1387)
==463==    by 0x437CE9: on_dns_packet (resolved-dns-transaction.c:1444)
==463==    by 0x4B2DC9B: source_dispatch (sd-event.c:3512)
==463==    by 0x4B2FB1F: sd_event_dispatch (sd-event.c:4077)
==463==    by 0x4B2FFFA: sd_event_run (sd-event.c:4138)
==463==    by 0x4B301D6: sd_event_loop (sd-event.c:4159)
==463==    by 0x464A24: run (resolved.c:92)
==463==    by 0x464B3C: main (resolved.c:99)
==463==  Address 0x5f409d0 is 32 bytes inside a block of size 72 free'd
==463==    at 0x48410E4: free (vg_replace_malloc.c:755)
==463==    by 0x418EDF: mfree (alloc-util.h:48)
==463==    by 0x4197E8: dns_query_candidate_free (resolved-dns-query.c:67)
==463==    by 0x4198B7: dns_query_candidate_unref (resolved-dns-query.c:70)
==463==    by 0x41A2E3: dns_query_unref_candidates (resolved-dns-query.c:337)
==463==    by 0x41C5FE: dns_query_cname_redirect (resolved-dns-query.c:1028)
==463==    by 0x41CA04: dns_query_process_cname_one (resolved-dns-query.c:1128)
==463==    by 0x41CA80: dns_query_process_cname_many (resolved-dns-query.c:1157)
==463==    by 0x40C0BD: bus_method_resolve_hostname_complete (resolved-bus.c:198)
==463==    by 0x41B312: dns_query_complete (resolved-dns-query.c:562)
==463==    by 0x41C1AC: dns_query_accept (resolved-dns-query.c:922)
==463==    by 0x41C2C4: dns_query_ready (resolved-dns-query.c:955)
==463==    by 0x41A162: dns_query_candidate_notify (resolved-dns-query.c:314)
==463==    by 0x434BD6: dns_transaction_complete (resolved-dns-transaction.c:437)
==463==    by 0x438995: dns_transaction_prepare (resolved-dns-transaction.c:1728)
==463==    by 0x43921D: dns_transaction_go (resolved-dns-transaction.c:1928)
==463==    by 0x419C7C: dns_query_candidate_go (resolved-dns-query.c:163)
==463==    by 0x41A143: dns_query_candidate_notify (resolved-dns-query.c:304)
==463==    by 0x434BD6: dns_transaction_complete (resolved-dns-transaction.c:437)
==463==    by 0x436A0F: dns_transaction_process_dnssec (resolved-dns-transaction.c:976)
==463==    by 0x4378C1: dns_transaction_process_reply (resolved-dns-transaction.c:1387)
==463==    by 0x437CE9: on_dns_packet (resolved-dns-transaction.c:1444)
==463==    by 0x4B2DC9B: source_dispatch (sd-event.c:3512)
==463==    by 0x4B2FB1F: sd_event_dispatch (sd-event.c:4077)
==463==    by 0x4B2FFFA: sd_event_run (sd-event.c:4138)
==463==    by 0x4B301D6: sd_event_loop (sd-event.c:4159)
==463==    by 0x464A24: run (resolved.c:92)
==463==    by 0x464B3C: main (resolved.c:99)
==463==  Block was alloc'd at
==463==    at 0x483E86F: malloc (vg_replace_malloc.c:380)
==463==    by 0x418F81: malloc_multiply (alloc-util.h:96)
==463==    by 0x419378: dns_query_candidate_new (resolved-dns-query.c:23)
==463==    by 0x41B42C: dns_query_add_candidate (resolved-dns-query.c:582)
==463==    by 0x41BB7A: dns_query_go (resolved-dns-query.c:762)
==463==    by 0x40CE3A: bus_method_resolve_hostname (resolved-bus.c:464)
==463==    by 0x4A84B86: method_callbacks_run (bus-objects.c:414)
==463==    by 0x4A87961: object_find_and_run (bus-objects.c:1323)
==463==    by 0x4A87FEE: bus_process_object (bus-objects.c:1443)
==463==    by 0x4AA3434: process_message (sd-bus.c:2964)
==463==    by 0x4AA3623: process_running (sd-bus.c:3006)
==463==    by 0x4AA4110: bus_process_internal (sd-bus.c:3226)
==463==    by 0x4AA41EF: sd_bus_process (sd-bus.c:3253)
==463==    by 0x4AA5343: io_callback (sd-bus.c:3604)
==463==    by 0x4B2DC9B: source_dispatch (sd-event.c:3512)
==463==    by 0x4B2FB1F: sd_event_dispatch (sd-event.c:4077)
==463==    by 0x4B2FFFA: sd_event_run (sd-event.c:4138)
==463==    by 0x4B301D6: sd_event_loop (sd-event.c:4159)
==463==    by 0x464A24: run (resolved.c:92)
==463==    by 0x464B3C: main (resolved.c:99)

Fixes #19376.
2021-05-14 23:18:10 +01:00
Zbigniew Jędrzejewski-Szmek b8f1045fe7 Revert "tools/make-directive-index: parallelize"
This reverts commit a2031de849.

The patch itself seems OK, but it exposes a bug in lxml or libxml2-2.9.12 which
was just released. This is being resolved in
https://gitlab.gnome.org/GNOME/libxml2/-/issues/255, but it might be while. So
let's revert this for now to unbreak our CI.

Fixes #19601.
2021-05-14 23:16:24 +01:00
Zbigniew Jędrzejewski-Szmek 7c22f07cbd meson: revert the change to unquote commands in add_install_script
Old meson fails with:
Element not a string: [<Holder: <ExternalProgram 'sh' -> ['/bin/sh']>>, '-c', 'test -n "$DESTDIR" || /bin/journalctl --update-catalog']

I'm doing it as a revert so that it's easy to undo the revert when we require
newer meson. The effect is not so bad, maybe a dozen or so lines about finding
'sh'.
2021-05-14 19:52:25 +02:00
Yu Watanabe 4301cb32f2 fix typo 2021-05-14 19:51:37 +02:00
Zbigniew Jędrzejewski-Szmek 6f1d18ae93 resolved: inline iterator variable declarations 2021-05-14 17:34:38 +02:00
Zbigniew Jędrzejewski-Szmek ebfdf45a20 resolved: drop pointless bitfield on a lone bool 2021-05-14 17:34:38 +02:00
Zbigniew Jędrzejewski-Szmek 421b1f0a41 resolved: optimize layout of DnsPacket
We obviously have lots of those, so even small savings add up.

Bitfields are dropped because they don't give any memory savings due to
alignment requirements (but would still require more complex to access).

/* size: 184, cachelines: 3, members: 28 */
/* sum members: 172, holes: 1, sum holes: 4 */
/* sum bitfield members: 4 bits (0 bytes) */
/* padding: 7 */
/* bit_padding: 4 bits */
    ↓
/* size: 176, cachelines: 3, members: 28 */
2021-05-14 17:34:38 +02:00
Zbigniew Jędrzejewski-Szmek 9aeb1a541b resolved: optimize layout of DnsTransaction
/* size: 296, cachelines: 5, members: 46 */
/* sum members: 278, holes: 5, sum holes: 17 */
/* sum bitfield members: 4 bits, bit holes: 1, sum bit holes: 4 bits */
    ↓
/* size: 280, cachelines: 5, members: 46 */
/* sum members: 278, holes: 1, sum holes: 1 */
/* sum bitfield members: 4 bits, bit holes: 1, sum bit holes: 4 bits */
2021-05-14 17:28:48 +02:00
Zbigniew Jędrzejewski-Szmek a6e890d493 resolved: rearrange DnsResourceRecord for better layout
The structure is rearranged to have less holes. Also fields in the union
are rearranged not to have holes (though most variants of the union still
have some padding at the end).

The full size does not decrease a lot, but the compiler should be able to
copy less bytes when it knows the specific type of the union.

Bitfields are dropped because they don't give any memory savings due to
alignment requirements (but would still require more complex to access).

The change from the this and previous commit:
/* size: 128, cachelines: 2, members: 13 */
/* sum members: 112, holes: 3, sum holes: 15 */
/* sum bitfield members: 2 bits, bit holes: 1, sum bit holes: 6 bits */
     ↓
/* size: 112, cachelines: 2, members: 13 */
/* sum members: 108, holes: 1, sum holes: 4 */
2021-05-14 17:15:53 +02:00
Zbigniew Jędrzejewski-Szmek 98e80bf9d2 resolved: use narrower types for label counts in rr
Order will be adjusted later to remove holes.
2021-05-14 17:03:30 +02:00
Zbigniew Jędrzejewski-Szmek 35aa30697a resolved: shorten dns_name_count_labels() 2021-05-14 16:48:13 +02:00
Zbigniew Jędrzejewski-Szmek 9079bd32ce resolved: pahole optimization for DnsQuery
Change from the last three commits:

/* size: 312, cachelines: 5, members: 46 */
/* sum members: 296, holes: 5, sum holes: 16 */
   ↓
/* size: 288, cachelines: 5, members: 46 */
/* sum members: 286, holes: 1, sum holes: 1 */

It's not a big difference, but we might have quite a few queries in flight,
so let' make this a bit more efficient.
2021-05-14 16:31:16 +02:00
Zbigniew Jędrzejewski-Szmek b4d1227858 resolved: make counter for redirects smaller
It can never go above 16, so let's use a narrower type.

Order will be adjusted later to remove holes.
2021-05-14 16:26:52 +02:00
Zbigniew Jędrzejewski-Szmek e1004d0aba resolved: make counter for n_auxiliary_queries smaller
It can never go above 64, so let's use a narrower type.

Order will be adjusted later to remove holes.
2021-05-14 16:26:26 +02:00
Lennart Poettering d47f96b4ef Merge pull request #19609 from keszybz/networkd-dhcp-man
Tweak man page ordering for DHCPv[46] options
2021-05-14 16:26:18 +02:00
Zbigniew Jędrzejewski-Szmek e3e64a1abd resolved: indent tables for readability
Some of the tables in resolved were already doing using this convention. This
makes the rest of them do so too.
2021-05-14 16:25:54 +02:00
Zbigniew Jędrzejewski-Szmek f86887af6e meson: print autodetected sbat fields
...
Message: sbat-distro (from ID): fedora
Message: sbat-distro-summary (from NAME): Fedora
Message: sbat-distro-url (from BUG_REPORT_URL): https://bugzilla.redhat.com/
...
2021-05-14 14:37:14 +02:00
Zbigniew Jędrzejewski-Szmek 0f4c4f3824 meson: call find_program() once and reuse the variable everywhere
Meson 0.58 has gotten quite bad with emitting a message every time
a quoted command is used:
Program /home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh found: YES (/home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program xsltproc found: YES (/usr/bin/xsltproc)
Configuring custom-entities.ent using configuration
Message: Skipping bootctl.1 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping journal-remote.conf.5 because HAVE_MICROHTTPD is false
Message: Skipping journal-upload.conf.5 because HAVE_MICROHTTPD is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping loader.conf.5 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
...

Let's suffer one message only for each command. Hopefully we can silence
even this when https://github.com/mesonbuild/meson/issues/8642 is
resolved.
2021-05-14 14:21:27 +02:00
Zbigniew Jędrzejewski-Szmek 6c55cefe2f man: reword descriptions of two DHCPv4 options 2021-05-14 11:48:33 +02:00
Zbigniew Jędrzejewski-Szmek c3696a9111 man: reorder items in [DHCPv4] and [DHCPv6] sections
The settings were listen in a completely random order, also different
between the v4 and v6 sections. Order by "options sent", "options received",
"communication settings" in both sections.

Also minor formatting changes are done, e.g. "=" is added in various places.
2021-05-14 11:48:33 +02:00