Commit Graph
2234 Commits
Author SHA1 Message Date
Franck Bui 6d7afa3b58 journald: make use of CLAMP() in cache_space_refresh() 2022-04-04 17:51:34 +02:00
Yu Watanabe 798931160e tree-wide: add a space after if, switch, for, and while 2022-04-01 22:48:42 +09:00
Yu Watanabe dd2396f20d Merge pull request #22861 from poettering/journald-sigterm
journald: don't let SIGTERM starve indefinitely
2022-03-26 00:27:47 +09:00
Lennart Poettering 19252b2548 journald: make sure SIGTERM handling doesn't get starved out
Fixes: #22642
2022-03-25 10:03:00 +01:00
Lennart Poettering 49615dbd81 journal-file: merge compress/seal bool args into a single flags param
Just some modernization/refactoring.

No change in behaviour, just let's do how we do things these days: use
flags param instead of list of bools.
2022-03-25 09:59:09 +01:00
Yu Watanabe de010b0b2e strv: make iterator in STRV_FOREACH() declaread in the loop
This also avoids multiple evaluations in STRV_FOREACH_BACKWARDS()
2022-03-19 08:33:33 +09:00
Yu Watanabe 03677889f0 list: declare iterator of LIST_FOREACH() in the loop 2022-03-19 08:10:29 +09:00
Jan Janssen b405e3aae1 test-journal-syslog: Add some valid priority cases 2022-03-16 14:50:12 +01:00
Jan Janssen 68da8adf54 test: Use TEST macros in more places 2022-03-16 14:50:12 +01:00
Yu Watanabe 3b591ebbd1 Merge pull request #22739 from mrc0mmand/list-boot-followup
A couple of follow-ups for #22721
2022-03-15 15:59:45 +09:00
Frantisek Sumsal f01aafd283 journal: make --reverse affect --list-boots
Fixes: #16274
2022-03-14 23:33:59 +01:00
Frantisek Sumsal 8e4b9a252b journal: use table_set_json_field_name() to override a column name
Pointed out in: https://github.com/systemd/systemd/pull/22721#discussion_r826014227
Follow-up for: 5a1355d848
2022-03-14 23:33:22 +01:00
Frantisek Sumsal faf20d4cca journal: reset previously set JSON flags
Make sure we reset the JSON format flags if the format option is used
multiple times, e.g. `journalctl -o json-format -o export`.

Pointed out in: https://github.com/systemd/systemd/pull/22721#discussion_r826018985
Follow-up for: 5a1355d848
2022-03-14 18:28:19 +01:00
Franck Bui 11ee11dbb3 journal: preserve acls when rotating user journals with NOCOW attribute set
When restoring the COW flag for journals on BTRFS, the full journal contents
are copied into new files. But during these operations, the acls of the
previous files were lost and users were not able to access to their old
journal contents anymore.
2022-03-14 18:03:02 +01:00
Frantisek Sumsal 5a1355d848 journal: convert --list-boots to a table
so it can be output as JSON as well.

```
$ build-san/journalctl --list-boots --file boot-test.journal
IDX BOOT ID                          FIRST ENTRY                 LAST ENTRY
 -3 39d66eb1925f4d01b8464d502650a714 Sat 2022-03-05 15:20:33 CET Sat 2022-03-05 16:19:21 CET
 -2 5dffeb08a27344d5ae9e2fc244bbcbc5 Fri 2022-03-11 17:23:57 CET Fri 2022-03-11 17:38:31 CET
 -1 c8ebd52915b642c39eda4bf00f864f79 Fri 2022-03-11 17:38:41 CET Fri 2022-03-11 20:03:46 CET
  0 00bcba97c7094fa88cc5d1cf2a389057 Sat 2022-03-12 20:39:08 CET Sat 2022-03-12 20:40:18 CET

$ build-san/journalctl --list-boots --file boot-test.journal -q
-3 39d66eb1925f4d01b8464d502650a714 Sat 2022-03-05 15:20:33 CET Sat 2022-03-05 16:19:21 CET
-2 5dffeb08a27344d5ae9e2fc244bbcbc5 Fri 2022-03-11 17:23:57 CET Fri 2022-03-11 17:38:31 CET
-1 c8ebd52915b642c39eda4bf00f864f79 Fri 2022-03-11 17:38:41 CET Fri 2022-03-11 20:03:46 CET
 0 00bcba97c7094fa88cc5d1cf2a389057 Sat 2022-03-12 20:39:08 CET Sat 2022-03-12 20:40:18 CET

$ build-san/journalctl --list-boots --file boot-test.journal -o json-pretty
[
        {
                "index" : -3,
                "boot_id" : "39d66eb1925f4d01b8464d502650a714",
                "first_entry" : 1646490033438495,
                "last_entry" : 1646493561047353
        },
        {
                "index" : -2,
                "boot_id" : "5dffeb08a27344d5ae9e2fc244bbcbc5",
                "first_entry" : 1647015837289036,
                "last_entry" : 1647016711595489
        },
        {
                "index" : -1,
                "boot_id" : "c8ebd52915b642c39eda4bf00f864f79",
                "first_entry" : 1647016721056382,
                "last_entry" : 1647025426397414
        },
        {
                "index" : 0,
                "boot_id" : "00bcba97c7094fa88cc5d1cf2a389057",
                "first_entry" : 1647113948506002,
                "last_entry" : 1647114018943637
        }
]
```

Resolves: #14625
2022-03-13 19:25:21 +01:00
Zbigniew Jędrzejewski-Szmek 2f492a739c meson: move files' closing brace to separate line 2022-03-03 12:14:13 +01:00
Zbigniew Jędrzejewski-Szmek f1b98127ff meson: do not use split() in file lists
The approach to use '''…'''.split() instead of a list of strings was initially
used when converting from automake because it allowed identical blocks of lines
to be used for both, making the conversion easier.

But over the years we have been using normal lists more and more, especially
when there were just a few filenames listed. This converts the rest.

No functional change.
2022-03-02 14:49:32 +01:00
Daan De Meyer ee48779e05 shared: Add more dlopen() tests
Add dlopen_dw(), dlopen_elf() and dlopen_pcre2() to the dlopen test.
To enable adding dlopen_pcre2(), we move pcre2-dlopen.h/c from
src/journal to src/shared.
2022-03-01 21:07:17 +00:00
Lennart Poettering d94e8a5064 journal-file: explicitly handle file systems that do not support hole punching
Apparently the error code fallocate() returns if hole punching is not
supported is not too well defined (man page just says "an error is
returned"), hence let's accept the usual set of errors, and the
normalize it to EOPNOTSUPP, and generate a clear error message in this
case.
2022-02-04 16:37:39 +01:00
Lennart Poettering 47497593fa journal-file: fix error handling of pread() in journald_file_punch_holes() 2022-02-04 16:37:20 +01:00
Lennart Poettering 5d04cec867 journal: when copying journal file to undo NOCOW flag, go via fd
We have the journal file open already, hence reference it via the fd
insted of the file name. After all, some other tool might have
renamed/deleted it already.

Let's not actually reuse the fd though, since we want a separate file
offset for the copying, hence just make it simply and reopen via
/proc/self/fd/.

Follow-up for d71ece3f0b
2022-02-03 11:34:24 +01:00
Daan De Meyer d02af6f33b journal: Rename JournaldFile to ManagedJournalFile
JournalFile and JournaldFile are hard to distinguish from each other.
Let's use ManagedJournalFile instead to make the distinction more clear.
2022-02-02 14:39:39 +00:00
Lennart Poettering e6d4a1106c journal-file: don't use pread() when determining where to append, use mmap as before
This partially undoes the effect of
ab6e257b3e.

Originally, we always used the mmap logic to determine the current end
of the file. ab6e257b3e changed this so
that we always used pread().

With this change we'll use pread() from the synchronization thread and
mmap otherwise.
2022-02-02 11:21:44 +09:00
Lennart Poettering e5d8473335 journal: various fixes to journal_file_read_object()
This fixes a bunch of issues:

pread() returns ssize_t, and returns errors in 'errno', handle that
correctly.

More importantly: it might incompletely read data in case we hit
EOF. Check for that, and handle it.

Finally, rename the function to journal_file_read_object_header(), since
it really doesn't read full objects, but only their headers.

Follow-up for: 117e21121e
2022-02-01 18:45:23 +01:00
Daan De Meyer d93abf465b journal: Truncate file instead of punching hole in final object
Instead of punching a hole in the final object if it's an entry array,
let's just truncate the file instead.
2022-01-27 15:21:37 +00:00
Daan De Meyer 3a787b5e29 journal: stat journal file after truncating
Let's make sure the data stored in last_stat is up-to-date after
truncating the journal file.
2022-01-27 14:46:59 +00:00
Daan De Meyer ec50313d4e journal: Pass data objects to journal_file_move_to_entry_..._for_data() functions
This reduces the number of calls to journal_file_move_to_object() which are heavy.
All call sites have easy access to the data object so this change doesn't end up
complicating things.
2022-01-26 20:16:00 +00:00
YmrDtnJu df4ec48f45 Fix journald audit logging with fields > N_IOVEC_AUDIT_FIELDS.
ELEMENTSOF(iovec) is not the correct value for the newly introduced parameter m
to function map_all_fields because it is the maximum number of elements in the
iovec array, including those reserved for N_IOVEC_META_FIELDS. The correct
value is the current number of already used elements in the array plus the
maximum number to use for fields decoded from the kernel audit message.
2022-01-21 23:12:45 +00:00
Daan De Meyer 12727c2bc2 journal: Copy holes when archiving BTRFS journal files
Previously, the holes we punched earlier would get removed when
copying the file. Let's enable the new COPY_HOLES flag to make
sure this doesn't happen.

In my test, this drops a 800MB btrfs journal (without compression)
to 720 MB.

Fixes #22087
2022-01-17 16:10:18 +00:00
Yu Watanabe 9e3e592946 Merge pull request #22098 from DaanDeMeyer/journal-corrupt-2
journal: Fixes for handling of corrupt entry objects
2022-01-14 21:23:32 +09:00
Daan De Meyer 4d6455c075 journal: Don't discard -b arg when followed by -e
Allowing -e to be used to view the last logs of a previous boot seems
like a useful feature so let's not discard -b options anymore when
followed by -e.

Fixes #22107
2022-01-14 01:10:22 +09:00
Daan De Meyer f2eceb5268 journal: Remove unused arguments from journal_file_next_entry_for_data() 2022-01-12 17:31:57 +00:00
Jan Janssen bbec46c817 meson: Use files() for fuzzers
Not having to provide the full path in the source tree is much
nicer and the produced lists can also be used anywhere in the source
tree.
2022-01-11 14:15:54 +01:00
Jan Janssen e1eeebbb11 meson: Use files() for tests
Not having to provide the full path in the source tree is much
nicer and the produced lists can also be used anywhere in the source
tree.
2022-01-11 14:15:54 +01:00
Daan De Meyer 5b4a634a03 journal: Log filename when we fail to write an entry 2022-01-05 11:17:47 +00:00
Daan De Meyer eff79e4e22 journal: Log a better message when we're rotating because a file is full
The previous message was confusing errors. When we're rotating because
we've reached the file size limit, let's log a better message.

Fixes #22007.
2022-01-05 11:17:44 +00:00
Daan De Meyer 94c5a83c6e journal: Handle partially read HashItem's when punching holes 2021-12-21 02:29:57 +09:00
Yu Watanabe d9338387d9 Merge pull request #21786 from keszybz/dirent-work
Make FOREACH_DIRENT and FOREACH_DIRENT_ALL declare the iterator variables
2021-12-17 04:07:35 +09:00
Zbigniew Jędrzejewski-Szmek 03f7ddf0bf meson: stop building out convenience libraries by default
The meson default for static_library() are:
build_by_default=true, install=false. We never interact with the
static libraries, and we only care about them as a stepping-stone towards
the installable executables or libraries. Thus let's only build them if
they are a dependency of something else we are building.

While at it, let's drop install:false, since this appears to be the default.

This change would have fixed the issue with lib_import_common failing
to build too: we wouldn't attempt to build it.

In practice this changes very little, because we generally only declare static
libraries where there's something in the default target that will make use of
them. But it seems to be a better pattern to set build_by_default to false.
2021-12-16 11:01:08 +01:00
Luca Boccassi 58f62d7079 Merge pull request #21784 from DaanDeMeyer/issue-21675
journal: Hole punching improvements
2021-12-15 22:38:00 +00:00
Zbigniew Jędrzejewski-Szmek af3b864d76 Define FOREACH_DIRENT through FOREACH_DIRENT_ALL
As in the previous commit, 'de' is used as the iterator variable name.
2021-12-15 22:50:00 +01:00
Daan De Meyer 24040269ee journal: Stop reading in increments of block size during hole punching
Let's not try to be overly clever here. This code path is not overly
performance sensitive and we should avoid trying to outsmart the kernel
without proper benchmarking.
2021-12-15 18:24:29 +01:00
Daan De Meyer d951ac5578 journal: Use 16kb buffer during hole punching
Let's use the same buffer size as used in as copy.h.
2021-12-15 18:22:17 +01:00
Daan De Meyer cdbba44878 journal: Correctly advance offset when iterating hash table entries
pread() is not guaranteed to completely fill up the given buffer with
data which we assumed until now. Instead, only increment the offset by
the number of bytes that were actually read.
2021-12-15 18:21:19 +01:00
Daan De Meyer a2799cc556 journal: Add a minimum hole size for hole punching
Let's not bother punching extremely small holes to avoid unnecessary
file fragmentation.
2021-12-15 18:17:22 +01:00
Zbigniew Jędrzejewski-Szmek c7f0d9e5ac tree-wide: make FOREACH_DIRENT_ALL define the iterator variable
The variable is not useful outside of the loop (it'll always be null
after the loop is finished), so we can declare it inline in the loop.
This saves one variable declaration and reduces the chances that somebody
tries to use the variable outside of the loop.

For consistency, 'de' is used everywhere for the var name.
2021-12-15 16:19:13 +01:00
Zbigniew Jędrzejewski-Szmek a2d7654f99 journactl: show info about journal range only at debug level (#21775)
The message that the "journal begins … ends …" has been always confusing to
users. (Before b91ae210e6 it was "logs begin … end …" which was arguably even
more confusing, but really the change in b91ae210e6 didn't substantially change
this.)

When the range shown is limited (by -e, -f, --since, or other options), it
doesn't really matter to the user what the oldest entries are, since they are
purposefully limiting the range. In fact, if we are showing the last few
entries with -e or -f, knowing that many months the oldest entries have is
completely useless.

And when such options are *not* used, the first entry generally corresponds to
the beginning of the range shown, and the last entry corresponds to the end of
that range. So again, it's not particularly useful, except when debugging
journalctl or such. Let's just treat it as a debug message.

Fixes #21491.
2021-12-15 06:30:15 +09:00
наб 2dd9285bac journalctl: have -f and -e imply no-value -b
Both of these take multiple seconds, best-case, to get the last few
entries, but do so instantly with -b
2021-12-10 07:27:09 +01:00
наб a96fc72dbf journal: succes[s]fully typo
As found by fossies codespell:
https://fossies.org/linux/test/systemd-main.bb916f3.211208.tar.gz/codespell.html
2021-12-08 13:23:56 +00:00
Vito Caputo 74fb5be62d journal-file: require MMapCache* for journal_file_open()
Previously the MMapCache* was optionally NULL, which open would
handle by creating a new MMapCache* for the occasion.

This produced some slightly circuitous refcount-handling code in
the function, as well as arguably creating opportunities for
weirdness where an MMapCache* was intended to be supplied but
happened to be NULL, which this magic would then paper over.

In any case, this was basically only being utilized by tests,
apparently just to avoid having to create an MMapCache.  So
update the relevant tests to supply an MMapCache and make
journal_file_open() treat a NULL MMapCache* as fatal w/assert.
2021-12-07 14:39:20 -08:00