Commit Graph
59582 Commits
Author SHA1 Message Date
Luca Boccassi 1e7fbbd4e1 Merge pull request #24546 from mrc0mmand/test-exec-deserialization-tweaks
A couple of tweaks for test-exec-deserialization
2022-09-03 00:15:45 +01:00
Lennart Poettering d96c7550a0 Merge pull request #24541 from poettering/bootspec-tweaks
bootspec: slightly stricter validation + process tries-left/tries-done counters in filenames
2022-09-02 21:29:31 +02:00
Luca Boccassi 31ed4b9147 mkosi: update to latest commit
Required to fix Debian testing/unstable builds, as resolved is
now in its own package
2022-09-02 19:46:54 +01:00
Frantisek Sumsal fda00958bb test: make pylint happy 2022-09-02 20:19:38 +02:00
Lennart Poettering f1ee066840 log: don't attempt to duplicate closed fd
if the console fd is not open we shouldn#t try to move it out of the 0…2
range.

Fixes: #24535
Alternative-for: #24537
2022-09-03 03:06:18 +09:00
Frantisek Sumsal 4aa84ef9a3 test: check for the output file in a loop
This should make the test faster on fast machines and more reliable on
slower/under-load machines, where the 4 sec sleep wasn't sometimes enough.

Spotted on C8S machines under load:

```
test_added_after (__main__.ExecutionResumeTest) ... FAIL
test_added_before (__main__.ExecutionResumeTest) ... ok
test_interleaved (__main__.ExecutionResumeTest) ... ok
test_issue_6533 (__main__.ExecutionResumeTest) ... ok
test_no_change (__main__.ExecutionResumeTest) ... ok
test_removal (__main__.ExecutionResumeTest) ... ok
test_swapped (__main__.ExecutionResumeTest) ... ok

======================================================================
FAIL: test_added_after (__main__.ExecutionResumeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test/test-exec-deserialization.py", line 101, in check_output
    with open(self.output_file, 'r') as log:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpjnec1dj4'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./test/test-exec-deserialization.py", line 150, in test_added_after
    self.check_output(expected_output)
  File "./test/test-exec-deserialization.py", line 104, in check_output
    self.fail()
AssertionError: None

----------------------------------------------------------------------
Ran 7 tests in 44.270s
```
2022-09-02 20:06:12 +02:00
Yu Watanabe 9fe6ce0a13 Merge pull request #24536 from yuwata/dissect-take-loop-device
dissect-image: introduce dissect_loop_device() which takes LoopDevice object
2022-09-03 03:05:57 +09:00
Yu Watanabe bad31660ed dissect-image: introduce dissect_loop_device() which takes LoopDevice object 2022-09-03 01:32:01 +09:00
Lennart Poettering 93b233e72a find-esp: call the right function
We want the parent dir here, let's fix that.
2022-09-02 18:12:25 +02:00
Lennart Poettering 95150f3f56 update TODO 2022-09-02 18:09:57 +02:00
Lennart Poettering 293e224056 bootspec: clarify we knowingly ignore all errors but ENOMEM 2022-09-02 18:09:57 +02:00
Lennart Poettering 7f5780edfb bootspec: properly parse tries done/tries left from bootspec file names
This has been a long-time omission in the userspace parser of bootspec
files. Correct that.

Fixes: #16457
2022-09-02 18:09:57 +02:00
Yu Watanabe 062d511776 loop-util: always set LoopDevice.node 2022-09-03 00:42:31 +09:00
Yu Watanabe 672780cde1 loop-util: rename loopdev -> node
Preparation for the next commit. No functional change.
2022-09-03 00:42:31 +09:00
Yu Watanabe b8a3f619ae core/swap: use devname_from_stat_rdev() 2022-09-03 00:42:31 +09:00
Yu Watanabe 778912a659 sd-device: introduce devname_from_stat_rdev() 2022-09-03 00:42:31 +09:00
Yu Watanabe 4fe46c349d sd-device: rename devpath_from_devnum() -> devname_from_devnum()
In sd-device, `devpath` is a kind of syspath without '/sys' prefix, e.g.
/devices/pci0000:00/0000:00:1c.4/0000:3c:00.0/nvme/nvme0/nvme0n1,
and `devname` is a path to the device node, e.g. /dev/nvme0n1.

Let's use the consistent name for the helper function.
2022-09-03 00:42:31 +09:00
Lennart Poettering f70de82f9f bootspec: rename "id" function parameters to "fname"
It's really the file name, so name it like that. Just some renaming.

(This is preparation for later work, to extract tries-done/tries-left
info from filenames)
2022-09-02 16:22:46 +02:00
Lennart Poettering 9ac2a89a8f bootspec: complain about loader.conf lines without parameter 2022-09-02 16:22:46 +02:00
Lennart Poettering d68c6bea5e bootspec: port type1 parser to log_syntax() 2022-09-02 16:22:46 +02:00
Lennart Poettering da8f277c09 bootspec: let's actually use the result of strstrip() for further parsing
Also, given we are looking for whitespace as separators, just pass NULL
as separators, extract_first_word() defaults to whitespace after all.
2022-09-02 14:25:48 +02:00
Lennart Poettering bb9133bb46 bootspec: simplify paths + insist they are normalized
Inspired by #23913, let's complain if people use paths with ".."
in Type #1 bootspec entries.

Let's prefix all paths with "/" if it is missing.

Let's simplify all paths.

let's refuse paths/warn with "..".

Fixes: #23913
2022-09-02 14:25:46 +02:00
Lennart Poettering 81e327c42c Merge pull request #24491 from poettering/compare-order
condition: unify operator parsing handling
2022-09-02 07:18:10 +02:00
Johannes Schauer Marin Rodrigues 3fa8a1148a sysusers: make sp_lstchg shadow field reproducible
If the environment variable SOURCE_DATE_EPOCH is set, use its value
instead of the current time.
2022-09-02 07:17:25 +02:00
Luca Boccassi dddec402de Merge pull request #24530 from poettering/loop-with-more-lock
a variety of loopback block device fixes
2022-09-02 00:04:32 +01:00
Lennart Poettering 10736074b4 mention ConditionKernelVersion= compat break in NEWS 2022-09-01 23:20:11 +02:00
Lennart Poettering 71a3ff036b condition: change operator logic to use $= instead of =$ for glob comparisons
So this is a bit of a bikeshedding thing. But I think we should do this
nonetheless, before this is released.

Playing around with the glob matches I realized that "=$" is really hard
to grep for, since in shell code it's an often seen construct. Also,
when reading code I often found myself thinking first that the "$"
belongs to the rvalue instead of the operator, in a variable expansion
scheme.

If we move the $ character to the left hand, I think we are on the safer
side, since usually lvalues are much more restricted in character sets
than rvalues (at least most programming languages do enforce limits on
the character set for identifiers).

It makes it much easier to grep for the new operator, and easier to read
too. Example:

before:
    ConditionOSRelease=ID=$fedora-*
after:
    ConditionOSRelease=ID$=fedora-*
2022-09-01 23:16:13 +02:00
Lennart Poettering 06219747f5 condition: change ConditionKernelVersion= so that =/!= mean literal string comparison, and ==/<> version comparison
The only reason to do this is to ensure uniformity with the other
options, that work like this, i.e. ConditionOSRelease= or
ConditionSecurity=.

This is a compatibility break, but a minor one, given that string
comparison and version comparison is mostly the same for equality and
inequality.
2022-09-01 23:16:13 +02:00
Lennart Poettering 666d314a74 condition: allow literal string compares in SMBIOS condition
This ensures that "=" and "!=" are now interpreted as literal string
compares, and "==" and "<>" are for version compares.

This is not a compat break, since the SMBIOS stuff has not been included
in any release yet.

Main reason to do this, is to be systematic with the other conditions
that check for text stuff.
2022-09-01 23:16:13 +02:00
Lennart Poettering c990742523 condition: allow fnmatch() matches in ConditionKernelVersion=
This is mostly to make things systematic, and brings no new
functionality, as not specifying any operator is identical to prefixing
with =$ anyway.
2022-09-01 23:16:13 +02:00
Lennart Poettering 4803b0bcaa compare: add macro for operator charset 2022-09-01 23:16:13 +02:00
Lennart Poettering 5f6b2d394b compare: propagate errors of fnmatch() as errors 2022-09-01 23:16:13 +02:00
Lennart Poettering 2e8fa6274d compare: support textual operators, and port analyze over to it 2022-09-01 23:16:13 +02:00
Lennart Poettering 6061c86693 compare: add two new operators "==" and "<>"
These two operators always indicate ordering comparisons, as opposed to
"=" and "!=" which depending on context mean literal string compares.

This is useful for ConditionOSRelease= for example, as this means
there's now always a way to do version compares.
2022-09-01 23:16:13 +02:00
Lennart Poettering 38c09fa008 compare: rework table in parse_compare_operator() to be array of structs
Let's change the lookup table to contain pairs of operator/strings,
instead of being indexed by operator.

The table isn't dense anymore, and this allows us to add alias strings
sooner or later.
2022-09-01 23:16:10 +02:00
Lennart Poettering 8daa674090 condition: allow fnmatch compares for ConditionOSRelease=
We support this for smbios matches, hence do so for /etc/os-release
matches too.
2022-09-01 23:15:14 +02:00
Lennart Poettering 57610982f7 compare: add flag for parse_compare_operator() to do equality/inequality comparison via simple string compares
This allows us to switch condition_test_osrelease() to use generic
version_or_fnmatch_compare() for executing the comparison.
2022-09-01 23:15:14 +02:00
Lennart Poettering 69f0a6091b compare: add a proper flags field for parse_compare_operator() 2022-09-01 23:15:14 +02:00
Lennart Poettering 97c7bed90a compare: drop use of FNM_EXTMATCH for now
None of our other fnmatch() calls make use of this, and the concept was
new to me at least. Given that this is only used for the recently added
SMBIOS field matches (and is not included in any release) let's disable
"extended" matches for now. We can certainly revisit this, and enable it
later if there is real demand, but if we do, we should probably add that
all over the place, not just for smbios matches.
2022-09-01 23:15:14 +02:00
Lennart Poettering 8bd2cf6e11 comapre: add generic implementation for comapring with verscmp+fnmatch 2022-09-01 23:15:14 +02:00
Lennart Poettering 650c4c8707 compare: move test_order() → compare-operator.[ch] 2022-09-01 23:15:14 +02:00
Lennart Poettering a8835c1190 condition: split out order operator enum
Let's move the operator enum into its own .c/.h file, so that we can
reuse it elsewhere, in particular systemd-analyze's compare-versions
logic.

Let's rename the concept CompareOperator, since it is nowadays
genericlaly about both order *and* fnmatch comparisons, hence just
naming it "order" is misleading.
2022-09-01 23:12:34 +02:00
Lennart Poettering 7cb349f0ca loop-util: make clearer how LoopDevice objects that do not encapsulate an actual loopback device are set up 2022-09-01 22:06:19 +02:00
Lennart Poettering 4c1d50e65c loop-util: lock the control device around clearing the loopback device and deleting it
This mirrors what we already do during allocation. We lock the control
device first, and then release the block device and then delete it.

This makes things substantially more robust as long all participants do
such locking: we won't attempt to delete a block device somebody else
already is using.
2022-09-01 22:06:19 +02:00
Lennart Poettering 87862cc2b4 loop-util: close lock fd before trying LOOP_CLR_FD in failure path
If the loopback device is open more than once LOOP_CLR_FD will fail,
hence close the lock fd first explicitly, so there's definitely only one
fd left.
2022-09-01 22:06:19 +02:00
Lennart Poettering 247738b4f5 loop-util: drop code to attach empty file
Back when I wrote this code I wasn't aware of BLKPG and what it can do.
Hence I came up with this hack to attach an empty file to delete all
partitions. But today we can do better with BLKPG: let's just explicitly
remove all partitions, and then try again.
2022-09-01 22:06:19 +02:00
Lennart Poettering 7f52206a2b loop-util: rework how we lock loopback block devices
Let's rework how we lock loopback block devices in two ways:

1. Lock a separate fd, instead of the main block device fd. We already
   did that for our internal locking when allocating loopback block
   devices, but do so for the exposed locking (i.e.
   loop_device_flock()), too, so that the lock is independent of the
   main fd we actually use of IO.

2. Instead of locking the device during allocation of the loopback
   device, then unlocking it (which will make udev run), and then
   re-locking things if we need, let's instead just keep the lock the
   whole time, to make things a bit safer and faster, and not have to
   wait for udev at all. This is done by adding a "lock_op" parameter to
   loop device allocation functions that declares the initial state of
   the lock, and is one of LOCK_UN/LOCK_SH/LOCK_EX. This change also
   shortens a lot of code, since we allocate + immediately lock loopback
   devices pretty much everywhere.
2022-09-01 22:05:32 +02:00
Lennart Poettering 234c2e16e5 dissect: drop partition removal code
This reverts a major chunk of 75d7e04eb4

Now that the loopback device code already destroys the partitions we
don't have to do this here anymore.

I am sure the right place to delete the partitions is in the loopback
code, since we really only should do that for loopback devices, see
bug #24431, and not on "real" block devices.

I am also not convinced dropping partitions the dissection logic doesn't
care about is a good idea, after all. The dissection stuff should
probably not consider itself the "owner" of the block devices it
analyzes, but take a more passive role: figure out what is what, but not
modify it.

Fixes: #24431
2022-09-01 22:05:10 +02:00
Lennart Poettering e8383058b2 dissect: don't remove partitions explicitly on umount anymore
We do that now automatically when releasing the loopback device, hence
we can drop the redundant try entirely.
2022-09-01 20:42:17 +02:00
Lennart Poettering 3a6ed1e19d loop-util: when clearing a loopback device delete partitions first, and take BSD lock
Whenever we release a loopback device, let's first synchronously delete
all partitions, so that we know that's complete and not done
asynchronously in the background. Take a BSD lock on the device while
doing so, so that udev won't make the devices busy while we do this.
2022-09-01 20:41:08 +02:00