Commit Graph
3572 Commits
Author SHA1 Message Date
Yu Watanabe 0073f6c623 mmap-cache: each Window is owned by MMapFileDescriptor
And MMapFileDescriptor always references MMapCache, hence it is not
necessary Window to have a reference to MMapCache.

This also renames the list fields 'by_fd' -> 'windows', to make them
consistent with the name of the head.

No functional change, just refactoring.
2023-10-04 19:53:27 +09:00
Yu Watanabe 40f5e6a941 mmap-cache: merge mmap_cache_fd_get() with try_context() and find_mmap()
The post operations (attach the found or new window to a context, update
the keep_always flag, and calculate the address) in three steps are
equivalent, and the two searching logics are quite simple.
Let's merge them to decrease the total lines.

No functional change, just refactoring.
2023-10-04 19:53:27 +09:00
Yu Watanabe 81598f5ead mmap-cache: merge window_matches() and window_matches_fd()
Let's drop meaningless optimization, and always check if the window is
owned by the expected fd.
2023-10-04 19:53:27 +09:00
Yu Watanabe 1ed867d309 mmap-cache: merge mmap_try_harder() with make_room()
The function make_room() is short and only used by mmap_try_harder().
Let's merge them with short comments.

No functional change, just refactoring.
2023-10-04 19:53:27 +09:00
Yu Watanabe 0096914023 sd-journal: drop unused argument for generic_array_bisect_plus_one() 2023-10-04 17:36:14 +09:00
Daan De Meyer 9c56d79e1c Merge pull request #29386 from yuwata/mmap-cache-cleanups-part1
mmap-cache: several cleanups (part1)
2023-10-04 08:49:24 +02:00
Daan De Meyer 180d5aeaa7 Merge pull request #29403 from yuwata/journal-decouple-journald-and-journal-remote
journal: drop ManagedJournalFile and decouple journald and journal-remote
2023-10-04 07:55:52 +02:00
Yu Watanabe 0558c86c78 sd-netlink: use secure_getenv()
Follow-up for 52afaee74b.

Fixes CID#1522326.
2023-10-04 07:48:30 +02:00
Yu Watanabe 31438511e0 sd-journal: drop unnecessary re-read of object
This reverts the following commits.
- a1640191b4
- 231741d355

These were done by my misunderstanding of the mmap cache behavior.

Also, this updates the comments added by
df04b9ed86.
2023-10-03 22:17:34 +01:00
Luca Boccassi 8867122783 Merge pull request #29390 from yuwata/sd-netlink-make-default-timeout-configurable
sd-netlink: make the default timeout configurable
2023-10-02 22:59:07 +01:00
Yu Watanabe 1d3ef11dfe journal: move several tests to libsystemd/sd-journal
This also renames test-journal-xyz for journald (rather than sd-journal)
to test-journald-xyz.
2023-10-02 10:52:45 +09:00
Yu Watanabe 52afaee74b sd-netlink: make the default timeout configurable by environment variable
On normal systems, triggering a timeout should be a bug in code or
configuration error, so I do not think we should extend the default
timeout. Also, we should not introduce a 'first class' configuration
option about that. But, making it configurable may be useful for cases
such that "an extremely highly utilized system (lots of OOM kills,
very high CPU utilization, etc)".

Closes #25441.
2023-10-01 12:41:10 +09:00
Yu Watanabe f5c615889a sd-netlink: make calc_elapse() return USEC_INFINITY when no timeout is requested
Then, timout_compare() becomes simplar, the timeout value becomes
consistent with what sd_netlink_get_timeout() provides.

This also drop unnecessary assignment of reply_callback.timeout after
the slot is dropped from the prioq.
2023-10-01 12:41:10 +09:00
Yu Watanabe 8fc4d1be77 mmap-cache: rebreak lines 2023-10-01 06:35:02 +09:00
Yu Watanabe b38977e5d3 mmap-cache: modernize free functions
No functional change, just refactoring.
2023-10-01 06:35:02 +09:00
Yu Watanabe 8ff0f36e00 mmap-cache: check if mmap protection mode is consistent when fd is already managed
Otherwise, MMapFileDescriptor with an unexpected protection mode may be
returned.
2023-10-01 06:35:02 +09:00
Yu Watanabe 8926a6a467 mmap-cache: modernize mmap_cache_add_fd()
- use hashmap_ensure_put(),
- use _cleanup_free_ attribute,
- use structured initializer.
2023-10-01 06:35:02 +09:00
Yu Watanabe 397caa8122 mmap-cache: use structured initializer 2023-10-01 06:35:02 +09:00
Yu Watanabe d37eeabc4f sd-journal: merge journal_file_next_entry_for_data() with generic_array_get_plus_one()
Because journal_file_next_entry_for_data() provides the first entry, while
journal_file_next_entry() actually provides the next entry of the input,
this also renames it to journal_file_move_to_entry_for_data().

Also, previously, on DIRECTION_UP the function did not fall back to the
'extra' entry when all entries linked in the chained array are broken.
This also fixes the issue, and now it fall back to the extra entry.
2023-09-30 20:10:15 +09:00
Yu Watanabe 3a23e41883 sd-journal: add/update comments 2023-09-30 20:10:15 +09:00
Yu Watanabe f85e79d3e5 sd-journal: add missing 'error' handling
When we reach an empty array, there are at least two possibilities:
- journal file is corrupted,
- invalid index is requested.

We cannot distinguish them here, let's simply return earlier.
2023-09-30 20:10:15 +09:00
Yu Watanabe fe6f2bd8a6 sd-journal: fix calculation of number of 'total' entries in the chained arrays
If there's corruption and we are going upwards, then the 'total'
must be decreased when we go to the previous array. However,
previously, we wrongly kept or increased the number. This fixes
the behavior.
2023-09-30 20:10:15 +09:00
Yu Watanabe b63f09e4ee sd-journal: merge two bump_entry_array() calls
No functional changes, just refactoring.
2023-09-30 20:10:15 +09:00
Yu Watanabe b7264911aa sd-journal: make bump_entry_array() return positive when a valid offset found
When it returns 0 offset, then the subsequent
journal_file_move_to_object() will fail. Let's return
generic_array_get() earlier in such situation.
2023-09-30 20:10:15 +09:00
Yu Watanabe eb6ba87344 sd-journal: drop redundant re-read of entry array object
This effectively reverts d9b61db922.

In the do-while loop, we do not read any other entry array object, hence
the current object is always in the mmap cache and not necessary to re-read it.
2023-09-30 20:10:15 +09:00
Daan De Meyer 4444564a95 Merge pull request #29193 from keszybz/path-util-adjustment
Make unit mangling follow paths
2023-09-29 11:33:12 +02:00
Yu Watanabe 304cb08f84 sd-journal: always fallback to find entry by realtime
Consider the following situation:
- There are two journal files (x and y), that contains entries for two boots (X and Y).
- The journal file x contains entries of the boot X, and y contains
  entries of Y.
- Nevertheless x does not contains entries of boot Y, it contains
  the _BOOT_ID= data object of boot Y. Of course, the data object is not
  referenced by any entries in the journal file x.

In such situation, when the current location of sd_journal is the head
of journal y, that is, the first entry of the boot Y,
sd_journal_previous() failed without this change, and
'journalctl --boot -NUM' for boot X failed.

Fixes #29275.
2023-09-26 13:26:22 +09:00
Yu Watanabe 1eede15851 sd-journal: refuse to write entry without boot ID
To make journal entries always contain valid boot ID.
2023-09-26 13:26:22 +09:00
Yu Watanabe b761ae0fca sd-journal: boot_id is always non-NULL
The two callers of journal_file_append_entry_internal() always pass
non-NULL boot ID.
2023-09-26 13:26:22 +09:00
Yu Watanabe 6ea51363c8 sd-journal: also verify tail_entry_boot_id and friends in journal_file_verify_header()
Then, we can drop the redundant check in journal_file_read_tail_timestamp().
2023-09-26 13:26:22 +09:00
Zbigniew Jędrzejewski-Szmek 660087dc9c tree-wide: add path_simplify_alloc() and use it
path_simplify_full()/path_simplify() are changed to allow a NULL path, for
which a NULL is returned. Generally, callers have already asserted before that
the argument is nonnull. This way path_simplify_full()/path_simplify() and
path_simplify_alloc() behave consistently.

In sd-device.c, logging in device_set_syspath() is intentionally dropped: other
branches don't log.

In mount-tool.c, logging in parse_argv() is changed to log the user-specified
value, not the simplified string. In an error message, we should show the
actual argument we got, not some transformed version.
2023-09-22 08:13:34 +02:00
Yu Watanabe c650d4cdef sd-journal: refuse entry objects with an empty boot ID
Otherwise, e.g. sd_journal_get_monotonic_usec() return an empty boot ID
when called for such a broken entry object.

Such a broken object may be stored when the system was not cleanly shutdown.

Fixes #29167.
2023-09-22 08:05:53 +02:00
Yu Watanabe afb81a0c05 sd-event: drop unnecessary call of sd_event_now() when requested relative time is zero 2023-09-21 11:03:37 +02:00
Yu Watanabe d08ff4506c sd-device: include missing alloc-util.h
unref_and_replace_full() is defined in alloc-util.h.
2023-09-21 14:28:55 +08:00
Luca Boccassi 2cb170376a Merge pull request #29235 from yuwata/sd-journal-reset-saved-direction-on-seek
sd-journal: reset saved direction on seek
2023-09-20 22:06:01 +01:00
Yu Watanabe 4aa33df845 sd-journal: also clear saved direction on seek
Otherwise, sd_journal_previous() -> real_journal_next(DIRECTION_UP) ->
next_beyond_location() wrongly handles that previously we hit EOF of
the file, and returns 0 without finding a matching entry.

Fixes #29216.
2023-09-21 01:59:21 +09:00
Mike Yuan 79de6eb1a7 tree-wide: explicitly compare return value of fd_is_fs_type with 0
According to our coding style.
2023-09-19 21:45:27 +08:00
Joerg Behrmann 7227dd816f treewide: fix typos
- mostly: usecase -> use case
- continously -> continuously
- single typos in docs/FILE_DESCRIPTOR_STORE.md
2023-09-19 10:05:38 +02:00
Mike Yuan e22c60a9d5 io-util: introduce loop_write_full that takes a timeout
Also drop do_poll as the use case is covered
by timeout.
2023-09-07 20:30:44 +08:00
Luca Boccassi 626a9eba09 Merge pull request #28988 from keszybz/sd128-arbitrary-values
Add sd-id128 and systemd-id128 functionality to do "app specific" with any "base"
2023-09-03 14:05:32 +01:00
Zbigniew Jędrzejewski-Szmek a53dc9b130 Merge pull request #29017 from msizanoen1/fix-onboot-rotate
journal: Relax boot ID and monotonic clock consistency checks
2023-09-02 14:26:44 +03:00
Zbigniew Jędrzejewski-Szmek fa96afb4c4 sd-id128: do not allow null 'app_id' param
If it is null, we get the 'base' param unchanged:
$ build/systemd-id128 show 00000000000000000000000000000001 \
  --app-specific=00000000000000000000000000000000
00000000000000000000000000000001

This is not good, because it breaks our promise that the base (usually either
machine-id or boot-id) cannot be derived from the result. Some application
using the library could use a null app id, inadvertently exposing the machine
or boot id. (This could happen because of forgotten initialization, or maybe
because the app id is configurable, and the user configures it wrongly.)

Note: the other way the secret is not exposed:
$ build/systemd-id128 show 00000000000000000000000000000000 \
  --app-specific=00000000000000000000000000000002
4f63080959264900b0d88d999dae2d3a

Normally systemd would not allow a null machine-id or boot-id, but we can let
the user do the calculation that if they want to.
2023-09-02 14:17:29 +03:00
Yu Watanabe 1d4c874d3c Merge pull request #24570 from topimiettinen/nft-sets-v2
network: firewall integration with NFT sets
2023-09-01 20:13:39 +09:00
msizanoen addcecf6f4 journal: Relax boot ID and monotonic clock consistency checks
The monotonic clock value consistency should only be checked if the boot
ID is the same as the last journal entry, and requiring that the current
boot ID be the same as the boot ID of the last entry is not necessary
for ensuring deterministic bisection as we already enforce strict
ordering of the real time clock value in a single journal file.

This fixes an issue where a new journal file is unnecessarily created
every boot, potentially wasting storage space.
2023-08-30 20:17:18 +07:00
msizanoen 2f766aca1e journal: Don't write to journal files without the new boot ID update behavior guarantee
With the old boot ID update behavior there's no way to compare the boot
ID of a new entry to be written with the boot ID of the last entry in
the journal in a performant manner, and therefore no way to determine if
monotonic clock value consistency checking is needed. Refuse to open
journal files without the new boot ID update behavior guarantee to
simplify consistency checking.
2023-08-30 20:16:45 +07:00
Zbigniew Jędrzejewski-Szmek 7c9de5d85d id128-util: add id128_from_string_not_null() 2023-08-29 17:06:47 +03:00
Zbigniew Jędrzejewski-Szmek 62b4bee9bc sd-id128: export sd_id128_get_app_specific()
We exported sd_id128_get_machine_app_specific() and
sd_id128_get_boot_app_specific(). But those operate on the current machine,
and sometimes the user might want to do the same calculation for a different
machine, or an earlier boot, etc. We expose the underlying implementation
to make this easy.
2023-08-29 17:06:47 +03:00
Zbigniew Jędrzejewski-Szmek 0f7735d7a9 sd-id128: avoid one memcpy
By aligning the output buffer, we can just use the result directly, no need to
copy stuff around.
2023-08-29 17:06:47 +03:00
Topi Miettinen 940965803e shared/firewall-util: parametrize table and set names
Parametrize table and set names for fw_nftables_add_masquerade_internal, rename
to nft_set_element_op_iprange to reflect more general usage. Export and use
nfproto_is_valid().

Remove also unused and obsolete NFPROTO_DECNET.
2023-08-23 19:53:13 +03:00
Zbigniew Jędrzejewski-Szmek c5ed77b220 libsystemd: use _NEG_ macros, adjust some comments
No functional change.
2023-08-16 12:52:56 +02:00