Commit Graph
40144 Commits
Author SHA1 Message Date
Luca Boccassi b25e08a752 Merge pull request #22998 from DaanDeMeyer/journal-compact-split
journal: Add compact mode
2022-10-07 16:19:03 +02:00
Daan De Meyer 721620e8a3 journal: Add --convert= command to journalctl
--convert writes the journal files read by journalctl to the given
location. The location should be specified as a full journal file
path (e.g. /a/b/c/converted.journal). The directory specifies where
the converted journal files will be stored. The filename specifies
the naming convention the converted journal files will follow.
2022-10-07 12:28:09 +02:00
Daan De Meyer e81710d3d0 journal: Store offsets to tail entry array objects in chain
Previously, we'd iterate an entry array from start to end every time
we added an entry offset to it. To speed up this operation, we cache
the last entry array in the chain and how many items it contains.
This allows the addition of an entry to the chain to be done in
constant time instead of linear time as we don't have to iterate
the entire chain anymore every time we add an entry.
2022-10-07 12:28:09 +02:00
Daan De Meyer 0e35afff1d journal: Introduce journal_file_data_payload()
journal_file_data_payload() retrieves the payload of a Data object,
optionally decompressing it and checking to see if matches a given
field. This function replaces all the decompression code in the sd-journal
codebase with a single function.

This commit should not introduce any changes in sd-journal behavior.
2022-10-07 12:28:05 +02:00
Daan De Meyer a9089a6604 journal: Use 32-bit entry item object offsets in compact mode
To do this, we move EntryItem out of journal-def.h and turn it into
a host only struct in native endian mode so we can still use it to
ship the necessary info around.

Aside from that, the changes are pretty simple, we introduce some
extra functions to access the right field depending on the mode and
convert all the other code to use those functions instead of
accessing the raw fields.

We also drop the unused entry item hash field in compact mode. We
already stopped doing anything with this field a while ago, now we
actually drop it from the format in compact mode.
2022-10-07 12:20:57 +02:00
Daan De Meyer 99daf3ce03 journal: Use 32-bit entry array offsets in compact mode
Before:

OBJECT TYPE      ENTRIES SIZE
Unused           0       0B
Data             3610336 595.7M
Field            5310    285.2K
Entry            3498326 1.2G
Data Hash Table  29	 103.1M
Field Hash Table 29      151.3K
Entry Array      605991  1011.6M
Tag              0	 0B
Total            7720021 2.9G

After:

OBJECT TYPE      ENTRIES SIZE
Unused           0	 0B
Data             3562667 591.0M
Field            3971    213.6K
Entry            3498566 1.2G
Data Hash Table  20	 71.1M
Field Hash Table 20	 104.3K
Entry Array	 582647  505.0M
Tag              0	 0B
Total            7647891 2.4G
2022-10-07 12:20:08 +02:00
Daan De Meyer d06727aec2 journal: Don't allocate objects above UINT32_MAX in compact mode
To allow storing offsets as 32-bit, we should never allocate objects
outside of the 32-bit range.
2022-10-07 12:20:08 +02:00
Daan De Meyer c92f1ebe5d journal: Run unit tests with and without compact mode enabled 2022-10-07 12:16:11 +02:00
Daan De Meyer 61297656c7 journal: Enable compact mode
We also add an environment variable $SYSTEMD_JOURNAL_COMPACT that
can be used to disable compact mode if needed (similar to
$SYSTEMD_JOURNAL_KEYED_HASH).
2022-10-07 11:28:12 +02:00
Daan De Meyer 87413812c9 journal: Add compact mode
This adds a new flag in preparation for incompatible journal changes
which will be gated behind this flag. The max file size of journal
files in compact mode is limited to 4 GiB.
2022-10-07 11:26:28 +02:00
Yu Watanabe b3fac19f1b network: free timer event source for NDisc when link is freed
Though, it should be already freed already freed in link_stop_engines()
-> ndisc_stop(). Just for safety.
2022-10-07 14:32:13 +09:00
Yu Watanabe 44085d63f0 network: drop unnecessary call of ndisc_vacuum()
After the commit 773024685b, DNS servers
or domains are dropped when their lifefime become zero. Hence, it is not
necessary to try to them when writing state file.

Of course, because of the accuracy of the timer event source or priority
of event sources, a possibility is introduced that a DNS server or domain
with zero lifetime is stored in the state file. However, such entry will
be dropped soon when the timer event source is triggered. Hence, that
should not cause any real issues.
2022-10-07 14:17:28 +09:00
Zbigniew Jędrzejewski-Szmek c76691d708 Merge pull request #24511 from martinetd/bpf1
libbpf: Add libbpf 1.0.0 compat
2022-10-06 19:01:33 +02:00
Arnaud Ferraris 5113436b05 repart: always honour --discard=no
Currently, even if `--discard=no` is passed to `systemd-repart`, the
`context_discard_gap_after()` function still runs normally, discarding
e.g. all blocks between the GPT and the start of the first partition.

This can lead to issues on some embedded devices, where this space
holds the bootloader and shouldn't be modified (creating a protective
partition there is not always possible due to the specifics of the boot
process of some ARM-based SoC's).

This commit ensures passing `--discard=no` would be enough to ensure
the bootloader isn't wiped in such cases.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@gmail.com>
2022-10-06 17:27:47 +02:00
Dominique Martinet 87e462f713 libbpf: add compat helpers for libbpf down to 0.1.0
- new symbols are available from libbpf 0.6.0 so could be used with
libbpf.so.0, but we're sure the old symbols will be there and this
simplifies code
- detection at runtime should always work, regardless of whether systemd
has been compiled with older or newer libbpf and runs with older or newer
libbpf
2022-10-06 21:33:55 +09:00
Aleksey Vasenev 2be1ae54ba ata_id: Fixed getting Response Code from SCSI Sense Data (#24921)
The Response Code is contained in the first byte of the SCSI Sense Data.
Bit number 7 is reserved or has a different meaning for some Response Codes
and is set to 1 for some drives.
2022-10-06 13:50:30 +09:00
Daan De Meyer 5a967e2887 Try to load libbpf.so.1 as well
libbpf had a soname bump. Our usage of libbpf is compatible with
both libbpf.so.0 and libbpf.so.1, so let's try to load from both.
2022-10-06 07:31:23 +09:00
Daan De Meyer 6b8085db68 libbpf: Remove use of deprecated APIs 2022-10-06 07:31:23 +09:00
Luca Boccassi e96180a88d Merge pull request #24877 from brauner/namespace_utils
namespace-util: add an initial set of tweaks
2022-10-04 21:59:48 +01:00
Daan De Meyer fd1ca01a86 repart: Drop usage of CHASE_WARN
CHASE_WARN only makes sense when CHASE_SAFE or CHASE_NO_AUTOFS are
used. repart uses neither so let's drop usage of CHASE_WARN.
2022-10-04 21:48:06 +01:00
Christian Brauner 4b00e738d5 mount-util: use in_same_namespace()
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org>
2022-10-04 18:51:30 +02:00
Christian Brauner f7a2dc3dd5 nspawn: use in_same_namespace() helper 2022-10-04 18:51:30 +02:00
Christian Brauner 2fe299a320 namespace-util: add in_same_namespace()
Add a helper for the canonical way to determine whether two namespaces
are identical.
2022-10-04 18:51:28 +02:00
Christian Brauner 241b15779b nsflags: replace namespace_flag_map with general namespace_info introduced earlier 2022-10-04 18:51:04 +02:00
Christian Brauner c3b9c418c0 namespace-util: add namespace_info 2022-10-04 18:46:27 +02:00
Lennart Poettering edcb46fcd7 Merge pull request #24900 from yuwata/network-ndisc-drop-outdated
network: ndisc: fix zero lifetime handling
2022-10-04 18:03:25 +02:00
Marius Vollmer 4e2baf2f0a bus: Process authentication after write
Once everything has been written, a server bus might now process a
pending "BEGIN" and start the bus.
2022-10-04 17:52:50 +02:00
Yu Watanabe e6766c538a udev-builtin-net_id: reading phys_port_name may be refused with EOPNOTSUPP
If reading the sysattr failed with such error, the whole operation in
net_id builtin command will fail, and the interface will not be renamed.

Fixes a bug introduced by 5bbcfbaa11.
2022-10-04 20:29:41 +09:00
Topi Miettinen a8ec0abe5a tmpfiles: fix assert
Oct 03 17:33:20 systemd-tmpfiles[872]: Assertion 'IN_SET(i->type, CREATE_BLOCK_DEVICE|CREATE_CHAR_DEVICE)' failed at src/tmpfiles/tmpfiles.c:1837, function create_device(). Aborting.

I think this is caused by the line:
b! /dev/private/smartmontools-dev/sda 0660 root disk - 8:0
2022-10-03 22:13:29 +01:00
Yu Watanabe 8aba7b839b network: ndisc: do not accept too many DNS servers or domains
If there exists multiple routers, then the previous logic may introduce
too many DNS servers or domains.
2022-10-04 02:52:49 +09:00
Yu Watanabe af2aea8bb6 network: ndisc: drop addresses and friends when RA with zero lifetime is received
Routers may send options with zero lifetime if previously announced
information is outdated. Hence, if we receive such messages, then we
need to drop relevant addresses or friends.

See e.g. https://www.rfc-editor.org/rfc/rfc4861#section-12.

Follow-up for 2ccada8dc4.
2022-10-04 02:52:46 +09:00
Yu Watanabe 0cf1fe8888 network: make sec_to_usec() map 0sec -> 0usec
Zero lifetime in RA is special, and we should not assign possibly very
short lifetime addresses or friends.

This should not change anything at least now, preparation for later
commits. Note, DHCPv4 and v6 code also uses it, but sd-dhcp-client and
sd-dhcp6-client already filtered messages with zero lifetime. Hence,
the change should not affect DHCP code.
2022-10-04 00:25:14 +09:00
Yu Watanabe 773024685b network: ndisc: also introduce timer event source to drop outdated settings
Otherwise, settings based on previously received RA messages will never
removed without receiving a new RA message.
2022-10-04 00:25:14 +09:00
Yu Watanabe 94e6d37c2b network: ndisc: drop outdated settings before processing RA message
Otherwise, e.g. if a router is replaced, then the previously received
settings may never dropped.

Follow-up for 2ccada8dc4.
2022-10-04 00:25:14 +09:00
Yu Watanabe 8d1babc51d network: ndisc: address_get() returns 0 on success
After the commit 3b6a3bdebf, address_get()
does not return 1.
2022-10-04 00:25:14 +09:00
Yu Watanabe d4b7631468 network: introduce {address,route}_remove_and_drop()
Preparation for later commits.
2022-10-04 00:25:14 +09:00
Yu Watanabe a115c60e0d network: ndisc: ignore prefix option with link-local prefix
See https://www.rfc-editor.org/rfc/rfc4861#section-4.6.2.
2022-10-04 00:25:14 +09:00
Yu Watanabe 167c7ae511 network: ndisc: read prefix earlier
No functional changes.
2022-10-04 00:25:14 +09:00
Daan De Meyer 898db9f3cf repart: Use chase_symlinks() instead of path_join()
Let's properly resolve symlinks and check if the source actually
exists.
2022-10-03 14:16:13 +01:00
Luca Boccassi 419f2742e7 repart: workaround spurious maybe-uninitialized warning
Build fails on Ubuntu Jammy
2022-10-03 10:26:31 +01:00
Yu Watanabe 0a8720c7f0 network: fix use-after-free
If the lifetime of the route is already expired, do not try to
configure it.

Fixes a use-after-free, as the Request object is already freed, thus, we
cannot use Route or Link stored in Request object.
2022-10-03 09:15:10 +02:00
Yu Watanabe 2ff7862bf2 resolve: drop remaining references for Monitor=
Follow-up for b25d819aee.
2022-10-03 08:26:05 +02:00
Thomas Hebb 2ccada8dc4 network: don't forget old RAs when a new one arrives
IPv6 Neighbor Discovery lets us autoconfigure a link's IPv6 addresses,
routes, DNS servers, and DNS search domains by listening for Router
Advertisement (RA) packets broadcast by one or more routers on the link.
Each RA can contain zero or more "options," each describing one piece of
configuration (e.g. a single route).

Currently, when we receive an RA from a router, we delete any addresses,
routes, etc. that originated from that router's previous RAs unless
they're also present as options in the new RA.

That behavior is a violation of RFC 4861[1]. In Section 9, the RFC
states that

    Senders MAY send a subset of options in different packets. ... Thus,
    a receiver MUST NOT associate any action with the absence of an
    option in a particular packet. This protocol specifies that
    receivers should only act on the expiration of timers and on the
    information that is received in the packets.

Several other passages in the RFC reiterate this. Section 6.2.3:

    A router MAY choose not to include some or all options when sending
    unsolicited Router Advertisements.

Section 6.3.4:

    Hosts accept the union of all received information; the receipt of a
    Router Advertisement MUST NOT invalidate all information received in
    a previous advertisement or from another source.

At least one consumer router in production today, the Google Nest Wifi,
often sends RAs that omit its global IPv6 prefix. When current versions
of systemd-networkd receive those RAs, they immediately delete the
interface's global IPv6 address, which breaks IPv6 connectivity.

Fix the issue by removing the invalidation logic entirely. It's not
needed at all, since we already invalidate addresses, routes, and DNS
configuration when the interface goes down or their lifetimes expire.

This fix does have the side effect of preventing changes to the .network
file (e.g. denylisted prefixes, whether to add routes from RAs) from
taking effect as soon as a new RA arrives. Instead, a full interface
reconfiguration is needed. But triggering those changes on RA receipt
was already rather arbitrary and out of the administrator's control, so
I think this change is fine.

commit 69203fba70 ("network: ndisc: remove old addresses and routes
after at least one SLAAC address becomes ready") introduced this
behavior. commit 50550722e3 fixed it partially, by preventing one
router's RAs from invalidating another router's configuration.

[1] https://www.rfc-editor.org/rfc/rfc4861

Fixes: 69203fba70 ("network: ndisc: remove old addresses and routes after at least one SLAAC address becomes ready")
2022-10-03 09:59:37 +09:00
Yu Watanabe 64ebc0da03 resolve: fix typo 2022-10-03 09:23:37 +09:00
Daan De Meyer f0e6cb196e repart: Take --root into account in read only filesystems shortcut 2022-10-03 08:00:07 +09:00
Daan De Meyer 8e52ed024d repart: Don't fail on missing verity sig partition
Also, provide a proper error message when we fail to find a verity
sibling partition.
2022-10-02 14:08:34 +01:00
Luca Boccassi 3cd1b8f403 Merge pull request #24878 from keszybz/condition-first-boot
Tweak condition first boot to use the same logic in pid1 and units
2022-10-01 19:30:21 +01:00
Zbigniew Jędrzejewski-Szmek 7cd43e34c5 manager: fix/change evaluation of ConditionFirstBoot
The code to evaluate the kernel command line option was busted because it
was doing 'return b == !!r' at a point where 'r > 0'. Thus we'd return "true"
in both cases:

$ SYSTEMD_PROC_CMDLINE=systemd.condition-first-boot build/systemd-analyze condition 'ConditionFirstBoot=true'
test.service: ConditionFirstBoot=true succeeded.
Conditions succeeded.
$ SYSTEMD_PROC_CMDLINE=systemd.condition-first-boot build/systemd-analyze condition 'ConditionFirstBoot=false'
test.service: ConditionFirstBoot=false succeeded.
Conditions succeeded.

We only use 'ConditionFirstBoot=true' in units, so this wasn't noticed.

But I think the logic is broken in general: the condition should evaluate as
true only during initial boot. If we rerun the units at later points, we should
not consider ConditionFirstBoot to be true.

Also, the first boot logic is also used in pid1 itself. AFAICT, for two
things: in first boot machine-id is initialized transiently (this allows
first-boot operations to be restarted if boot fails), and preset-all is
executed. But this logic was different and separate from the logic to
evaluate ConditionFirstBoot. The distinction is abolished, and the operations
in pid1 now use the same logic as ConditionFirstBoot, which means that the
kernel command line option is checked, and condition_test_first_boot()
just tests whether pid1 thinks we're in first boot.

This makes things easier to grok for the user: there's just one condition for
"first boot" and it applies to both pid1 and units.
2022-10-01 11:35:46 +02:00
James Hilliard 0d73603b25 Revert "bpf: fix is_allow_list section"
GCC was modified to use the same default section as LLVM, as such
this is no longer required.

Details:
https://github.com/gcc-mirror/gcc/commit/a0aafbc324aa90421f0ce99c6f5bbf64ed163da6

This reverts commit e8b1e9cf10.
2022-10-01 13:24:50 +09:00
Luca Boccassi 7b8e0c3e9f Merge pull request #24829 from yuwata/blockdev-new-from-fd
blockdev-util: introduce block_device_new_from_fd() and _path()
2022-09-30 20:07:49 +01:00