Commit Graph
4080 Commits
Author SHA1 Message Date
Lennart Poettering 37ef2fc9f7 Merge pull request #18863 from keszybz/cmdline-escaping
Escape command lines properly
2021-05-07 17:29:39 +02:00
Zbigniew Jędrzejewski-Szmek d0f14a6cf4 Merge pull request #19134 from poettering/outbound-special-hostname
introduce a new synthetic hostname "_outbound" that maps to "the" local IP address
2021-05-07 17:15:22 +02:00
Lennart Poettering 2708160ccd fileio: optionally, return discovered path of file in search_and_fopen() 2021-05-07 16:43:26 +02:00
Lennart Poettering e9b88a6d4e string-util: add strextendf() helper, that allows extending some allocated string via a format string
It's not going to be efficient if called in inner loops, but it's oh so
handy, and we have some code that does this:

   asprintf(&p, "%s…", b, …);
   free(b);
   b = TAKE_PTR(p);

which can now be replaced by the quicker and easier to read:

   strextendf(&p, "…", …);
2021-05-07 09:10:59 +02:00
Yu Watanabe 3d396b2837 test: drop log_trace() in test-random-util
It provides almost no inoformation. Moreover, it massively increases
logs, and firefox cannot show the results of Ubuntu CIs due to this.
2021-05-06 12:46:31 +09:00
Zbigniew Jędrzejewski-Szmek 73ce91a05a Make unit_name_to_instance() return UnitNameFlags
The function returns non-negative UnitNameFlags on success, and negative
errno on error. In the past we kept the return type as int because of those
negative return values. But nowadays _UNIT_NAME_INVALID == -EINVAL. And if
we tried to actually return something that doesn't fit in the return type,
the compiler would throw an error. By changing to the "real" return type,
we allow the debugger to use symbolic representation for the variables.
2021-05-05 15:08:48 +02:00
Zbigniew Jędrzejewski-Szmek 510c7a953e test-process-util: add test that prints all cmdlines 2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek 99009ed0f4 basic/process-util: add mode where posix shell escape is used for quoting
The new flag is not used, except in tests, so no functional change yet.

This way, the command as shown can be copied-and-pasted into the shell
in more cases. For simple cases, shell quoting with "" is enough. But
$'' is needed when there are control characters in the command.
2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek 07468a16e4 test-process-util: run fewer getpid() tests
Significant time was spent in the getpid() measurement code, which is not very
important.  So let's optimize this a bit by running the slower version less
times, and only running both tests a lesser amount of times unless slow tests
are enabled.

This gives the better accuracy then before in slow mode, and still reasonable
accuracy in fast mode without a noticable slowdown.
2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek daceaabe1f test-process-util: add more debug logging but hide most of it by default
It makes little sense to always print the stuff that is fully deterministic
and verified by asserts. It can be opted-in with $SYSTEMD_LOG_LEVEL when
developing the tests or debugging a failure.
2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek 61977664e9 basic/process-util: allow quoting of commandlines
Since the new functionality is controlled by an option, this causes no change
in output yet, except tests.

The login in the old branch of !(flags & PROCESS_CMDLINE_QUOTE) is essentially
unmodified. But there is an important difference in behaviour: instead of
unconditionally reading the whole virtual file, we now read only 'max_columns'
bytes. This makes out code to write process lists quite a bit more efficient
when there are processes with long command lines.
2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek 82208a9949 test-utf8: hide most output by default
Unless one is working on the code, there is little reason to write most
of the output. So let's hide it unless requested with SYSTEMD_LOG_LEVEL=debug.
2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek fc96e5c053 basic/escape: allow truncation mode where "…" is always appended
So far we would append "…" or "..." when the string was wider than the specified
output width. But let's add a mode where the caller knows that the string being
passed is already truncated.

The condition for jumping back in utf8_escape_non_printable_full() was
off-by-one. But we only jumped to that label after doing a check with a
stronger condition, so I think it didn't matter. Now it matters because we'd
output the forced ellipsis one column too early.
2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek b19f211698 basic/escape: flagsify xescape_full() 2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek d12ccbc302 test-fileio: modernization 2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek ad0e687c07 basic/fileio: add a mode to read_full_virtual_file() where not the whole file is read 2021-05-05 13:59:23 +02:00
Zbigniew Jędrzejewski-Szmek 0089ab0800 basic/escape: escape control characters, but not utf-8, in shell quoting
The comment in the code said that so far this didn't matter, but I want to use
shell quoting in more places where this will make a difference. So control
characters are now escaped. Normal utf-8 characters are passed through, it
is 2021 after all and pretty much everyone is (or should be) using utf-8.

While touching the code, change 'char *r' → 'char *buf', in line with modern
style.
2021-05-05 12:12:42 +02:00
Zbigniew Jędrzejewski-Szmek 566d06ae50 basic/escape: always escape newlines in shell_escape()
shell_escape() is mostly used for mount paths and similar, where we assume
no newlines are present in the string. But if any were ever present, we
should escape them. So let's simplify the code by making this unconditional.
2021-05-05 12:12:42 +02:00
Zbigniew Jędrzejewski-Szmek 1129cd8a71 basic/escape: add mode where empty arguments are still shown as ""
For variables, FOO= is OK. But when quoting positional arguments, we want to
use something with quotes ("", '', or even $'') for an empty string.
2021-05-05 12:12:10 +02:00
Zbigniew Jędrzejewski-Szmek b086a89695 Merge pull request #19453 from keszybz/consider-instance-aliases
Consider instance aliases
2021-05-05 11:23:42 +02:00
Zbigniew Jędrzejewski-Szmek 32464e18b4 test-unit-file: enable colors
There's quite a lot of output and it's much easier to grok this way.
2021-05-04 13:36:05 +02:00
Lennart Poettering 16ecdf3c80 Merge pull request #19403 from nmeyerhans/dmi-entries
virt: differentiate Amazon EC2 bare-metal from VM instances using SMBIOS data
2021-05-03 17:48:13 +02:00
Bertrand Jacquin b6eca3731d virt: detect Amazon EC2 Nitro instance
Amazon EC2 Nitro hypervisor is technically based on KVM[1], which
systemd-detect-virt identify propely from CPUID. However the lack of
CPUID on aarch64 (A1, T4 instance type) prevents a correct
identification, impacting hostnamectl and systemd-random-seed. Instead
it's possible to identify virtualization from DMI vendor ID.

Prior to this commit:
  # hostnamectl
     Static hostname: n/a
  Transient hostname: ip-10-97-8-12
           Icon name: computer
          Machine ID: 8e3772fbcfa3dd6f330a12ff5df5a63b
             Boot ID: b7b7e2fe0079448db664839df59f9817
    Operating System: Gentoo/Linux
              Kernel: Linux 5.4.69-longterm
        Architecture: arm64

After this commit:
  # hostnamectl
     Static hostname: n/a
  Transient hostname: ip-10-97-8-12
           Icon name: computer-vm
             Chassis: vm
          Machine ID: 8e3772fbcfa3dd6f330a12ff5df5a63b
             Boot ID: bd04da57084e41078f20541101867113
      Virtualization: amazon
    Operating System: Gentoo/Linux
              Kernel: Linux 5.4.69-longterm
        Architecture: arm64

[1] https://aws.amazon.com/ec2/faqs/
2021-04-30 09:31:34 -07:00
Yu Watanabe df7ee95913 udev: do not store inotify fd in a global variable
When manager_exit() or manager_free() is called, the global variable in
udev-watch.c is not set '-1'. Of course, that is safe, as the event source
for the inotify fd is unref()ed in manager_exit() and manager_free().
But let's not store fd globally.
2021-04-30 19:21:18 +09:00
Julia Kartseva 28b76fc82a tests: add test program for SocketBind{Allow|Deny}=
Verify that service exited correctly if valid ports are passed to
SocketBind{Allow|Deny}=
Use `ncat` program starting a listening service binding to a specified
port, e.g.
"timeout --preserve-status -sSIGTERM 1s /bin/nc -l -p ${port} -vv"
2021-04-26 16:26:28 -07:00
Julia Kartseva a8e5eb1788 core: add socket-bind cgroup mask harness
Standard cgroup harness for bpf feature.
2021-04-26 16:21:59 -07:00
Frantisek Sumsal 732a487fad Merge pull request #19383 from keszybz/test58-fixes
Fixes for TEST-58-REPART and ExecStart deserialization logic
2021-04-26 19:16:44 +02:00
Zbigniew Jędrzejewski-Szmek 35243b7736 test-unit-serialize: add a very basic test that command deserialization works
We should test both serialization and deserialization works properly.
But the serialization/deserialization code is deeply entwined with the
manager state, and I think quite a bit of refactoring will be required before
this is possible. But let's at least add this simple test for now.
2021-04-26 16:15:26 +02:00
Lennart Poettering a1fdbcbe3d nss-myhostname: expose the "outbound" IP addresses under the synthetic "_outbound" hostname
I found myself often looking for a quick way to determine "the local IP
address", and then being lost in the "ip addr" output to find for the
right one to use. This is supposed to help a bit with that. Let's
introduce a new special hostname "_outbound" with semantics similar to
"_gateway" that resolves to addresses that are the closest I could come
up with that maps to "the" local IP address.
2021-04-23 12:02:11 +02:00
Lennart Poettering 54e6f97bc9 local-addresses: add helper for determining local "outbound" IP addresses
This adds a small helper, similar in style to local_addresses() and
local_gateways() that determines the local "outbound" addresses.

What's an "outbound" address supposed to be? The local IP addresses that
are the most likely used for outbound communication. It's determined
by using connect() towards the default gws on an UDP socket, and then
reading the address of the socket this caused it to be bound to.

This is not the "public" or "external" IP address of the local system,
and is not supposed to be. It's just the local IP addresses that are
likely the ones going to be used by the local IP stack for
communication with other hosts.
2021-04-23 12:01:41 +02:00
David Tardon 8adbad370f test-install-root: add test for unknown WantedBy= target 2021-04-23 07:28:37 +02:00
David Tardon cd228002cc test-install-root: create referenced targets 2021-04-23 07:28:37 +02:00
Igor Zhbanov 4c54768c97 journald: Retry if posix_fallocate returned -1 (EINTR)
On some conditions (particularly when mobile CPUs are going to sleep),
the posix_fallocate(), which is called when a new journal file is allocated,
can return -1 (EINTR). This is counted as a fatal error. So the journald
closes both old and journals, and simply throwing away further incoming
events, because of no log files open.

Introduce posix_fallocate_loop() that restarts the function in the case
of EINTR. Also let's make code base more uniform by returning negative
values on error.

Fix assert in test-sigbus.c that incorrectly counted positive values as
success. After changing the function return values, that will actually work.

Fixes: #19041

Signed-off-by: Igor Zhbanov <i.zhbanov@omprussia.ru>
2021-04-21 23:08:35 +02:00
Zbigniew Jędrzejewski-Szmek 64b21afc72 sd-id128: add convenience functions to compare multiple sd_id128_t
Similar to sd_bus_error_has_names() that was added in
2b07ec316a.

It is made inline in the hope that the compiler will be able to optimize
all the va_args boilerplate away, and do an efficient comparison when
the arguments are all constants.
2021-04-21 17:51:24 +02:00
Lennart Poettering 4a62257d68 dissect: ignore udev database entries from before the loopback attachment
This tries to shorten the race of device reuse a bit more: let's ignore
udev database entries that are older than the time where we started to
use a loopback device.

This doesn't fix the whole loopback device raciness mess, but it makes
the race window a bit shorter.
2021-04-20 17:20:38 +02:00
Lennart Poettering 75dc190d39 dissect: ignore old uevents when waiting for loopback partition scan
Let's drop all monitor uevent that were enqueued before we actually
started setting up the device.

This doesn't fix the race, but it makes the race window smaller: since
we cannot determine the uevent seqnum and the loopback attachment
atomically, there's a tiny window where uevents might be generated by
the device which we mistake for being associated with out use of the
loopback device.
2021-04-20 17:14:10 +02:00
Zbigniew Jędrzejewski-Szmek a626cb15c0 basic/log: assert that 0 is not passed as errno, except in test code
Let's assert if we ever happen to pass 0 to one of the log functions.
With the preceding commit to return -EIO from log_*(), passing 0 wouldn't
affect the return value any more, but it is still most likely an error.
The unit test code is an exception: we fairly often pass the return value
to print it, before checking what it is. So let's assert that we're not
passing 0 in non-test code. As with the previous check for %m, this is only
done in developer mode. We are depending on external code setting
errno correctly for us, which might not always be true, and which we can't
test, so we shouldn't assert, but just handle this gracefully.

I did a bunch of greps to try to figure out if there are any places where
we're passing 0 on purpose, and couldn't find any.
The one place that failed in tests is adjusted.

About "zerook" in the name: I wanted the suffix to be unambiguous. It's a
single "word" because each of the words in log_full_errno is also meaningful,
and having one term use two words would be confusing.
2021-04-14 17:07:06 +02:00
Zbigniew Jędrzejewski-Szmek 47350c5fb6 meson: simplify the BUILD_MODE conditional
Using a enum is all nice and generic, but at this point it seems unlikely that
we'll add further build modes. But having an enum means that we need to include
the header file with the enumeration whenerever the conditional is used. I want
to use the conditional in log.h, which makes it hard to avoid circular imports.
2021-04-14 16:19:54 +02:00
Julia Kartseva b57162aafb tests: add unit file tests for BPFProgram=
- Pin trivial bpf programs to bpf filesystem, compose BPFProgram= option
string and pass it to a unit. Programs store `0` in r0 BPF register for
denying action, e.g. drop a packet.
- Load trivial BPF programs
- Test is skipped if not run under root or if can not lock enough
memory.
- For egress and ingress hooks, test BPFProgram= option along with
with IP{Egress|Ingress}FilterPath=, expected result should not depend on
which rule is executed first.
Expected results for BPF_CGROUP_INET_INGRESS:
5 packets transmitted, 0 received, 100% packet loss, time 89ms

For BPF_CGROUP_INET_SOCK_CREATE:
ping: socket: Operation not permitted
2021-04-09 20:28:47 -07:00
Julia Kartseva 506ea51b48 core: add bpf-foreign cgroup mask and harness
Add CGROUP_MASK_BPF_FOREIGN to CGROUP_MASK_BPF and standard cgroup
context harness.
2021-04-09 20:28:47 -07:00
Zbigniew Jędrzejewski-Szmek 22344fcfb9 Merge pull request #19243 from bluca/lgtm
Fix various issues reported by LGTM
2021-04-09 13:12:41 +02:00
Zbigniew Jędrzejewski-Szmek d0b3039837 Merge pull request #19226 from keszybz/reenable-maybe-unitialized-warning
meson: re-enable -Wmaybe-uninitialized
2021-04-08 20:29:05 +02:00
Yu Watanabe ecb4b08c2e test: move check of nat table existence
As test_v4() with iptables backend will be called after nftables tests.

Follow-up for afbcd90552.
2021-04-08 14:39:53 +02:00
Luca Boccassi b01339f74f test-extract-word: add a couple more corner cases 2021-04-08 13:07:31 +01:00
Zbigniew Jędrzejewski-Szmek 7756528e9b tree-wide: use the same comment for work-around initializations
This should make it easier to remove those warnings when the compiler
gets smarter. Not sure if I got them all...

Double space before the comment start to make it easier to separate from the
preceding line.
2021-04-07 16:04:22 +02:00
Luca Boccassi 727e63e33e basic: add path_equal_filename helper 2021-04-07 14:03:49 +01:00
Luca Boccassi 65d09d575c Bump test-random-util timeout
It is using log_trace, which we enabled in the CI, so now it's
slower and it is timing out. Bump the timeout from 30s to 120s.
2021-04-07 09:32:42 +02:00
Anita Zhang afbcd90552 test-firewall-util: skip if iptables nat table does not exist 2021-04-06 08:01:27 +02:00
Anita Zhang ea460d7964 meson: link with libm for math functions
Fixes this error I got building on F33:
  /usr/bin/ld: test-random-util.p/src_test_test-random-util.c.o: undefined
  reference to symbol 'sqrt@@GLIBC_2.2.5'
  /usr/bin/ld: /usr/lib64/libm.so.6: error adding symbols: DSO missing
  from command line
2021-04-01 18:45:48 -07:00
Zbigniew Jędrzejewski-Szmek 9e53c10a0f Flagsify EscapeStyle and make ESCAPE_BACKSLASH_ONELINE implicit
I want to tweak behaviour further, and that'll be easier when "style"
is converted to a bitfield.

Some callers used ESCAPE_BACKSLASH_ONELINE, and others not. But the
ones that didn't, simply didn't care, because the argument was assumed to
be one-line anyway (e.g. a service name). In environment-generator, this
could make a difference. But I think it's better to escape the newlines
there too. So newlines are now always escaped, to simplify the code and
the test matrix.
2021-04-01 12:46:24 +02:00