Commit Graph
3126 Commits
Author SHA1 Message Date
Luca Boccassi 697f082697 Merge pull request #24853 from poettering/resolved-monitor-fixes
resolved: various monitor fixes
2022-09-30 15:46:13 +01:00
Luca Boccassi ace212f577 portablectl: add --force attach/detach
Allows to skip check that ensures units must not be running.

I have a use case that would use reattach, except the orchestrator
is using a non-standard versioning scheme, so image matching cannot
work. As a workaround, need to be able to detach and then attach
manually, without stopping the units to avoid extended downtimes
and loss of FD store.
2022-09-30 13:25:31 +01:00
Lennart Poettering b968890a87 test: rework resolved monitoring test
Let's remove some sleep loops, and instead:

1. Use Type=notify to wait until "resolvectl monitor" successfully
   installed its monitor, so that we know that queries enqueued later
   will definitely be seen.

2. Use "grep -m1" to watch "journalctl -f" output to wait precisely for
   the RR data we want to see, and immediately exit.

This shortens code quite a bit, and should make it more robust.
2022-09-30 14:24:41 +02:00
Frantisek Sumsal bb0f817abf test: wait until the unit finishes before checking the log
Otherwise we might read an incomplete log and fail:

```
test_added_after (__main__.ExecutionResumeTest) ... FAIL
test_added_before (__main__.ExecutionResumeTest) ... ok
test_interleaved (__main__.ExecutionResumeTest) ... ok
test_issue_6533 (__main__.ExecutionResumeTest) ... ok
test_no_change (__main__.ExecutionResumeTest) ... ok
test_removal (__main__.ExecutionResumeTest) ... ok
test_swapped (__main__.ExecutionResumeTest) ... ok

======================================================================
FAIL: test_added_after (__main__.ExecutionResumeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/./test/test-exec-deserialization.py", line 152, in test_added_after
    self.check_output(expected_output)
  File "/build/./test/test-exec-deserialization.py", line 107, in check_output
    self.assertEqual(output, expected_output)
AssertionError: 'foo\n' != 'foo\nbar\n'
  foo
+ bar

----------------------------------------------------------------------
Ran 7 tests in 27.470s
```
2022-09-30 18:47:09 +09:00
Frantisek Sumsal 842a9d5f91 test: kill the machine on oops/panic/soft_lockup
Otherwise the machine will hang on the panic until the timeout happens,
which might waste quite a considerable amount of time in certain cases.
2022-09-30 18:46:30 +09:00
Frantisek Sumsal 9b2e40a885 Merge pull request #24865 from yuwata/udevadm-tweaks
udevadm: fix misleading error message
2022-09-30 00:01:48 +02:00
Yu Watanabe ad8c442a0b test-64-udev-storage: use wait command instead of hackish "udevadm lock true"
Otherwise, "udevadm lock true" may lock a block device earlier than
"udevadm lock sfdisk &".
2022-09-30 03:42:05 +09:00
Frantisek Sumsal fa21a2cf9f test: bump the reattach timeout when running w/ plain QEMU
As it might sometimes take slightly longer without the acceleration:

```
[  176.805681] testsuite-29.sh[534]: + cp /usr/share/app1.raw /tmp/app1_2.raw
[  176.885365] testsuite-29.sh[534]: + timeout 30 portablectl reattach --now --runtime --extension /tmp/app1_2.raw /usr/share/minimal_1.raw app1
[  177.053358] portablectl[993]: (Matching unit files with prefixes 'app1'.)
[  177.138770] kernel: loop0: detected capacity change from 0 to 2965504
[  177.343137] kernel: loop1: detected capacity change from 0 to 4096
...
[  201.932062] systemd[1]: app1.service: Deactivated successfully.
[  202.009310] systemd[1]: Stopped app1.service.
[  202.053776] systemd[1]: app1.service: Consumed 2.183s CPU time.
[  202.125061] systemd[1]: Stopping app1.service...
[  202.611760] systemd[1]: Starting modprobe@dm_mod.service...
[  202.851031] systemd[1]: Starting modprobe@dm_verity.service...
[  202.909352] systemd[1]: Starting modprobe@loop.service...
[  203.198918] systemd[1]: Starting app1.service...
[  207.145494] kernel: audit: type=1130 audit(1663770336.105:428): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=testsuite-29 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
[  207.652545] systemd[1]: testsuite-29.service: Main process exited, code=exited, status=124/n/a
[  207.665088] systemd[1]: testsuite-29.service: Failed with result 'exit-code'.
[  207.830522] systemd[1]: Failed to start testsuite-29.service.
...
[  208.889449] script1.sh[1035]: ID="centos"
[  208.889449] script1.sh[1035]: VERSION_ID="8"
[  208.889449] script1.sh[1035]: SYSEXT_SCOPE=portable
[  208.889449] script1.sh[1035]: PORTABLE_PREFIXES=app1
...
[  214.155097] systemd[1]: app1.service: Deactivated successfully.
```

Spotted in Ubuntu CI and CentOS CI.

Follow-up to 706c9a30ac.
2022-09-30 00:16:45 +09:00
Frantisek Sumsal 5ad1513845 test: wrap ls and stat to make it work w/ sanitizers in specific cases
When `/etc/nsswitch.conf` uses `systemd` together with `[SUCCESS=merge]`,
`ls -l` will pull in `libnss_systemd` causing `SIGABRT`, as `ls` is not
instrumented (by default):

```
-bash-5.1# strace -f -e %file ls -l /dev
execve("/usr/bin/ls", ["ls", "-l", "/dev"], 0x7ffc3bb211c8 /* 24 vars*/) = 0
...
openat(AT_FDCWD, "/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=1896, ...}, AT_EMPTY_PATH) = 0
newfstatat(AT_FDCWD, "/etc/nsswitch.conf", {st_mode=S_IFREG|0644, st_size=359, ...}, 0) = 0
openat(AT_FDCWD, "/etc/group", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=965, ...}, AT_EMPTY_PATH) = 0
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=10779, ...}, AT_EMPTY_PATH) = 0
openat(AT_FDCWD, "/usr/lib/libnss_systemd.so.2", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0755, st_size=16195176, ...}, AT_EMPTY_PATH) = 0
openat(AT_FDCWD, "/usr/lib/libasan.so.8", O_RDONLY|O_CLOEXEC) = 3
...
readlink("/proc/self/exe", "/usr/bin/ls", 4096) = 11
open("/proc/self/cmdline", O_RDONLY)    = 3
open("/proc/self/environ", O_RDONLY)    = 3
==620==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=620, si_uid=0} ---
+++ killed by SIGABRT (core dumped) +++
Aborted (core dumped)
```

This also happens with `stat`. Let's add both `ls` and `stat` to the "wrap list"
to work around this.

Spotted on Arch Linux.
2022-09-28 22:39:18 +01:00
Frantisek Sumsal 07faa4990f test: fix a copy-paste error 2022-09-28 14:03:03 +02:00
Luca Boccassi 0e26016e3d resolved notifications: follow-up fixes
Further review comments from: https://github.com/systemd/systemd/pull/22845
2022-09-27 22:34:17 +01:00
Yu Watanabe a13231a904 Merge pull request #24832 from mrc0mmand/more-TEST-64-tweaks
A couple of performance tweaks for TEST-64 under QEMU
2022-09-28 02:26:55 +09:00
Frantisek Sumsal 998fb5c5e3 test: use fewer partitions/LVs when running with plain QEMU 2022-09-27 15:10:18 +02:00
Frantisek Sumsal 90743af65d test: make the symlink helpers a bit more quiet
and show only errors/warnings.
2022-09-27 15:10:18 +02:00
Frantisek Sumsal d6ef4ebbda test: ignore tty* devices when checking device units
This lower the runtime of `check_device_units()` in a plain QEMU VM from
~45 seconds to ~25 seconds.
2022-09-27 15:10:18 +02:00
Frantisek Sumsal dd2b99ee7f test: lower the # of iterations when running with plain QEMU 2022-09-27 15:10:18 +02:00
Frantisek Sumsal d9e1cb288f test: support open-iscsi >= 2.1.2
Since open-iscsi 2.1.2 [0] the initiator name should be generated via
a one-time service instead of distro package's post-install scripts.
However, some distros still use this approach even after this patch,
so prefer the already existing initiatorname.iscsi file if it exists.

[0] https://github.com/open-iscsi/open-iscsi/commit/f37d5b653f9f251845db3f29b1a3dcb90ec89731
2022-09-27 13:24:32 +02:00
Luca Boccassi 542e6eb38d Merge pull request #24799 from poettering/initrd-ftw
use "initrd" rather than "initial RAM disk" or "initramfs" to refernce the concept
2022-09-23 20:43:15 +01:00
Daan De Meyer 354dc913c5 Merge pull request #24635 from DaanDeMeyer/repart-verity-sig
repart: Add support for generating verity sig partitions
2022-09-23 18:53:04 +02:00
Daan De Meyer b456191d3c repart: Add support for generating verity sig partitions 2022-09-23 16:15:37 +02:00
Lennart Poettering dd5533801b Merge pull request #24700 from poettering/ssh-creds
support easy provisioning for SSH key of root user
2022-09-23 16:01:09 +02:00
Lennart Poettering 32e2767080 tree-wide: use the term "initrd" at most places we so far used "initramfs"
In most cases we refernced the concept as "initrd". Let's convert most
remaining uses of "initramfs" to "initrd" too, to stay internally
consistent.

This leaves "initramfs" only where it's relevant to explain historical
concepts or where "initramfs" is part of the API (i.e. in
/run/initramfs).

Follow-up for: b66a6e1a58
2022-09-23 15:10:53 +02:00
Daan De Meyer 3dd73ea77e dissect: Process verity sig partitions if a root hash is specified
If a root hash is specified, we should be checking that it matches
the root hash in the verity signature partition, so let's not skip
processing of the verity signature partitions if a root hash is
specified.
2022-09-23 12:24:09 +02:00
Daan De Meyer 3c5f7ec4ca test: Install openssl 3 extra library dependencies 2022-09-23 12:24:09 +02:00
Daan De Meyer bc259e2338 repart: Rename verity integration test definition files 2022-09-23 12:24:09 +02:00
Yu Watanabe 0b0cdb1652 doc: drop remaining references to LGTM.com 2022-09-23 18:29:22 +09:00
Lennart Poettering aebdd3f3d7 test: add test case for new ':' uid/gid/access modifier in tmpfiles.d 2022-09-23 09:31:54 +02:00
Yu Watanabe 767bc538c5 test-network: fix matching string
This partially reverts 5515f2169c.
As the commit changes 'networkctl list', not 'networkctl status'.
2022-09-23 10:43:17 +09:00
Lennart Poettering 6ca0016398 measure: allow pre-calculating PCR values for multiple boot phases 2022-09-22 16:54:48 +02:00
Lennart Poettering add0c9b8bf Merge pull request #24768 from keszybz/table-not-available-2
Adjust table n/a text in more places
2022-09-22 14:17:40 +02:00
Zbigniew Jędrzejewski-Szmek 5515f2169c networkctl: use "-" for empty fields
This looks nicer. Some fields were already using "-" as fallback, e.g. Speed,
so this makes things more consistent too.
2022-09-21 14:48:28 +02:00
Zbigniew Jędrzejewski-Szmek 639405b934 hostnamectl,localectl: use "(unset)" in empty fields
"n/a" is more ambiguous: not available or not set or maybe we didn't check it.
Let's just say directly that the field is not set.
2022-09-21 14:48:28 +02:00
Daan De Meyer 0178ee6779 Merge pull request #24522 from yuwata/core-device-drop-nonexistent-devlink-unit
core/device: drop nonexistent devlink units
2022-09-21 10:30:51 +02:00
Yu Watanabe fb227a3de1 test-17-udev: test that device units for nonexistent devlink are removed
For issue #24518.
2022-09-21 05:58:40 +09:00
Yu Watanabe ede0fe0271 test-64-storage: add test for renaming lvm volume 2022-09-21 05:58:40 +09:00
Yu Watanabe 9c126b2c92 test-64-udev-storage: check device units 2022-09-21 05:58:40 +09:00
Frantisek Sumsal 761b1d8314 test: don't overwrite existing $QEMU_OPTIONS 2022-09-20 22:40:02 +02:00
Yu Watanabe d64b44be52 test: add testcase for link priority 2022-09-19 06:08:58 +09:00
Yu Watanabe d3fcb73678 test-50-dissect: do not fail test on cleanup
These paths are read-only mount points. Hence, we cannot remove the
contents.
2022-09-17 21:36:18 +09:00
Yu Watanabe 759b4b4a2a test-50-dissect: check mount destination instead of mount source
And rebreak long lines.

This should take no effective change, just refactoring.
2022-09-17 21:36:18 +09:00
Frantisek Sumsal 5a634f2ec0 test: drop the use of tempfile.mktemp()
and use `uuid.uuid4()` instead to generate a sufficiently pseudo-random
file name.

Resolves: https://github.com/systemd/systemd/security/code-scanning/142
2022-09-13 21:03:04 +02:00
Frantisek Sumsal 2a843ec832 test: drop forgotten format()
Follow-up to fda00958bb.
2022-09-13 21:03:04 +02:00
Yu Watanabe ee27501225 test: add testcase for udev-watch 2022-09-12 01:36:06 +09:00
Frantisek Sumsal e3cccd3c2b test: make the resolved notifications check a bit more robust
Let's parse the resolved JSON notifications via `jq` and check them in a
bit more "controlled" manner - e.g. until now the `grep` was checking just
a one gigantic JSON string, as all received notifications via the
varlink socket are terminated by a NUL character, not a newline.

Also, as the notification delivery is asynchronous, retry the check
a couple of times if it fails (spotted in C8S jobs):

```
[ 2891.935879] testsuite-75.sh[36]: + : '--- nss-resolve/nss-myhostname tests'
[ 2891.935988] testsuite-75.sh[36]: + run getent -s resolve hosts ns1.unsigned.test
[ 2891.936542] testsuite-75.sh[177]: + getent -s resolve hosts ns1.unsigned.test
[ 2891.937499] testsuite-75.sh[178]: + tee /tmp/tmp.pqjNvbQ2eS
[ 2891.939977] testsuite-75.sh[178]: 10.0.0.1        ns1.unsigned.test
[ 2891.940258] testsuite-75.sh[36]: + grep -qE '^10\.0\.0\.1\s+ns1\.unsigned\.test' /tmp/tmp.pqjNvbQ2eS
[ 2891.942235] testsuite-75.sh[189]: + grep -qF '[10,0,0,1]'
[ 2891.942577] testsuite-75.sh[188]: + grep -aF ns1.unsigned.test /tmp/notifications.txt
[ 2891.943978] systemd[1]: testsuite-75.service: Child 36 belongs to testsuite-75.service.
[ 2891.944112] systemd[1]: testsuite-75.service: Main process exited, code=exited, status=1/FAILURE
[ 2891.944215] systemd[1]: testsuite-75.service: Failed with result 'exit-code'.
```
2022-09-11 14:29:34 +02:00
Frantisek Sumsal 7906b790e4 test: install jq into all images 2022-09-11 14:16:34 +02:00
Yu Watanabe 706c9a30ac test-29-portable: set timeout for 'portablectl reattach'
Then, the test can fail earlier than the timeout of the whole test
specified by $QEMU_TIMEOUT=.

This is useful when we try to run the test multiple times.

Workaround for issue #24147.
2022-09-10 08:00:44 +00:00
Suraj Krishnan cb456374e0 Implement DNS notifications from resolved via varlink
* The new varlink interface exposes a method to subscribe to DNS
resolutions on the system. The socket permissions are open for owner and
group only.
* Notifications are sent to subscriber(s), if any, after successful
resolution of A and AAAA records.

This feature could be used by applications for auditing/logging services
downstream of the resolver. It could also be used to asynchronously
update the firewall. For example, a system that has a tightly configured
firewall could open up connections selectively to known good hosts based
on a known allow-list of hostnames. Of course, updating the firewall
asynchronously will require other design considerations (such as
queueing packets in the user space while a verdict is made).

See also:
https://lists.freedesktop.org/archives/systemd-devel/2022-August/048202.html
https://lists.freedesktop.org/archives/systemd-devel/2022-February/047441.html
2022-09-09 09:22:57 +01:00
Yu Watanabe 3b19e16056 test-58-repart: add test case for issue #24553 2022-09-09 05:53:01 +09:00
Frantisek Sumsal 8432b0cd20 Merge pull request #24351 from poettering/pcr-sign
support for signed TPM2 PCR policies in cryptsetup/cryptenrolls/credentials
2022-09-08 19:07:04 +00:00
Frantisek Sumsal f53d56f1a3 test: check the numa-test.service status directly
In a couple of recent CI runs I noticed TEST-36 failing due to a missed
service exit notification and a subsequent fail of the `grep` command:

```
[  257.112153] H systemd[1]: Started numa-test.service.
[  257.114343] H systemd[899]: numa-test.service: Failed to set NUMA memory policy: Invalid argument
[  257.118270] H systemd[899]: numa-test.service: Failed at step NUMA_POLICY spawning /bin/sleep: Invalid argument
[  257.126170] H systemd[1]: Bus private-bus-connection: changing state RUNNING → CLOSING
[  257.130290] H systemd[1]: numa-test.service: Failed to send unit change signal for numa-test.service: Connection reset by peer
[  257.131567] H systemd[1]: Received SIGCHLD from PID 899 ((sleep)).
[  257.132870] H systemd[1]: Got disconnect on private connection.
[  257.134299] H systemd[1]: systemd-journald.service: Got notification message from PID 498 (FDSTORE=1)
[  257.135611] H systemd[1]: systemd-journald.service: Added fd 38 (n/a) to fd store.
[  257.136999] H systemd[1]: systemd-journald.service: Received EPOLLHUP on stored fd 38 (stored), closing.
[  257.366996] H testsuite-36.sh[536]: + stopJournalctl
[  257.366996] H testsuite-36.sh[536]: + local unit=init.scope
[  257.366996] H testsuite-36.sh[536]: + echo 'Force journald to write all queued messages'
[  257.366996] H testsuite-36.sh[536]: Force journald to write all queued messages
[  257.366996] H testsuite-36.sh[536]: + journalctl --sync
[  257.488642] H systemd-journald[498]: Received client request to rotate journal.
[  257.520821] H testsuite-36.sh[536]: + journalctl -u init.scope --cursor-file=jounalCursorFile
[  257.981399] H testsuite-36.sh[536]: + pid1StopUnit numa-test.service
[  257.984533] H testsuite-36.sh[536]: + systemctl stop numa-test.service
[  258.173656] H systemd[1]: Bus private-bus-connection: changing state AUTHENTICATING → RUNNING
[  258.180710] H systemd[1]: numa-test.service: Trying to enqueue job numa-test.service/stop/replace
[  258.182424] H systemd[1]: Added job numa-test.service/stop to transaction.
[  258.185234] H systemd[1]: numa-test.service: Installed new job numa-test.service/stop as 738
[  258.187017] H systemd[1]: numa-test.service: Enqueued job numa-test.service/stop as 738
[  258.239930] H testsuite-36.sh[536]: + grep 'numa-test.service: Main process exited, code=exited, status=242/NUMA' journal.log
```

Let's mitigate this by checking the test service exit status directly
instead of relying on the notification.
2022-09-08 15:00:26 +00:00