This also adds bus_assert_return() as an variant of assertion.
With this change, more code blocks are checked by coccinelle,
thus the test become slightly slower. So the timeout is extended.
Also, as checking exec_invoke() takes so long, let's skip it.
Note, previously the function was skipped due to parse error caused by
unknown cleanup attributes. So, the coverage is not decreased by the
skip.
Drop the -fundamental suffix from src/fundamental/ headers in favor of names
that match their src/basic/ or src/shared/ counterparts (e.g.
macro-fundamental.h -> macro.h, assert-fundamental.h -> assert-util.h,
cleanup-fundamental.h -> cleanup-util.h). Rename src/basic/{btrfs,label}.{c,h}
to use the -util suffix to match the existing shared/btrfs-util and
shared/label-util siblings. Rename src/shared/mkdir-label.{c,h} to mkdir.{c,h}
and src/shared/tmpfile-util-label.{c,h} to tmpfile-util.{c,h} to match the
corresponding src/basic names.
This saves us from having to come up with separate names for files that do
the same thing across tiers, and it makes it easier to move stuff between
src/fundamental, src/basic and src/shared: consumers just #include "foo.h"
and pick up whichever tier their -I path resolves to first, so call sites
don't need to be updated when an API moves between layers.
Where a higher-tier wrapper exists (e.g. src/basic/macro.h wrapping
src/fundamental/macro.h), the wrapper uses an explicit "../fundamental/foo.h"
or "../basic/foo.h" relative include for the lower-tier header. We can't use
GCC's #include_next directive for this — when the wrapper is reachable both
via same-dir-as-source lookup and via -I (e.g. -Isrc/shared) for the
directory it lives in, #include_next advances by exactly one slot in libcpp's
internal directory chain and lands on the same physical directory it was
already in, never reaching the lower-tier sibling (see make_cpp_dir() in
gcc/libcpp/files.cc:1986).
To make sure the right headers are always picked up, the include directories
are reordered so that e.g. src/shared always takes priority over src/basic and
similar for the other directories.
Co-developed-by: Claude Opus 4.7 <noreply@anthropic.com>
We have code like `size_t max_size = sizeof(*private)` in three
places. This is evaluated at compile time so its safe to use. However
the new pointer-deref checker in coccinelle is not smart enough to know
this and will flag those as errors. To avoid these false positives
we have some options:
1. Reorder so that we do:
```C
size_t max_size = 0;
assert(private);
max_size = sizeof(*private);
```
2. Use something like `size_t max_size = sizeof(*ASSERT_PTR(private));`
3. Place the assert before the declaration
4. Workaround coccinelle via SIZEOF(*private) that we can then hide
via parsing_hacks.h
5. Fix coccinelle (OCaml, hard)
6. ... somehting I missed?
None of these is very appealing. I went for (4) but happy about
suggestions.
This commit runs the check-coccinelle checker scripts with the
parsing_hacks.h. Because this was missing before there were some
issues that did not get flagged.
While at it it also adds some missing cleanup attributes and
iterators to get better results. Its a bit sad that there is no
(easy/obvious) way to detect when new things are needed for
parsing_hacks.h
Our coccinelle/check-pointer-deref.cocci checker has a special
case for `assert(pidref_is_set(param))`. It turns out we can
generalize this and catch the following:
- iovec_is_set
- sd_dhcp_duid_is_set
- sd_dhcp_client_id_is_set
The fix in 8f1751a111 made me wonder if we could automatically detect
when pointers are accessed but when this might not be safe. Systemd
is already using a lot of `assert(dst)` and this change now forces
us to use them.
So this commit (ab)uses coccinelle to flag any pointer parameter
dereference not preceded by assert(param), ASSERT_PTR(param), or an
explicit NULL check. It adds integration into meson as a new "coccinelle"
test suite (just like clang-tidy) and is run in CI. The check is not
perfect but seems a reasonable heuristic.
For this RFC commit it is scoped to a subset, it excludes 25 dirs right
now and includes around 100. About 300 warnings left. Busywork that I am
happy to do if there is agreement that it is worth it.
With this in place we would have caught the bug from 8f1751a111 in CI:
```
FAIL: check-pointer-deref.cocci found issues in systemd/src/boot:
diff -u -p systemd/src/boot/measure.c /tmp/nothing/measure.c
--- systemd/src/boot/measure.c
+++ /tmp/nothing/measure.c
@@ -312,7 +312,6 @@ EFI_STATUS tpm_log_tagged_event(
if (err != EFI_SUCCESS)
return err;
- *ret_measured = true;
return EFI_SUCCESS;
}
```
This also adds a new POINTER_MAY_BE_NULL() for the cases when the
called function will do the NULL check (like `iovec_is_set()`).
Use Coccinelle's "depends on" directive to exclude files from certain
transformations. This should make them a bit simpler and possibly
faster, since we don't have to shell out to Python.
Unfortunately, this works only for file/directory exclusions. For
function and other more complex exclusions we still need to use Python,
at least for now.
Also, completely drop the file exclusion for man/ in the xsprintf
transformation, since we filter out everything under man/ before we even
run Coccinelle (in run-coccinelle.sh).
Currently, parsing zlib.h on Fedora (and possibly others) causes spatch
to fail with an assertion. Let's work around that by defining two extra
macros in our Coccinelle parsing hacks.
Even though these are in our tree, we should still treat them as
system includes which helps various tools (clangd, iwyu, ...) understand
that these are system includes and <> should be used instead of "".
Turns out Coccinelle can handle compound literals just fine, the parsing
errors were caused by incorrectly parsed macros in code before the
literals, so let's just provide simplified versions for such macros.
The parsing error in `Type *foo[ELEMENTSOF(bar)] = {};` is actually
harmless; it occurs only when creating an array of pointers for a type
that's in an external header and it occurs only on the first parser's
pass, subsequent passes resolve the type correctly.
Also, unset ENABLE_DEBUG_HASHMAP, so Coccinelle doesn't expand the
hashmap debug macros.
As for the remaining FIXMEs, I opened a couple of issues in the
Coccinelle upstream to see if they can be fixed there (or at least
properly analyzed).
Turns out I _really_ underestimated the impact of
--include-headers-for-types, as it significantly reduces both runtime
and storage penalties. For example, on my machine the runtime of
uncached run goes down from ~15 minutes to ~2 minutes, and similarly the
total storage needed by the cache goes from ~15 GiB down to ~3 GiB.
Drop the original macro file, since it's not needed anymore thanks to
resolving includes properly, but introduce a similar file -
parsing_hacks.h - that helps Coccinelle in some specific corner cases.
This eliminates most of the outstanding parsing errors in source files.
The remaining ones are limitations of the parsing engine (see the FIXMEs
in pasing_hacks.h) and need further investigation.
Since Coccinelle is originally a kernel tool, it doesn't search the
system include path by default for header files. Without this we're
missing a lot of types provides by stdlib (and other libraries we make
use of).
Turns out that the original way we did things was quite broken, as it
skipped a _lot_ of code. This was because we just threw everything into
one pile and tried to spatch it, but this made Coccinelle sad, like when
man page examples redefined some of our macros, causing typedef
conflicts.
For example, with a minimal reproducer that defines a cleanup macro in
two source files, Coccinelle has no issues when spatch-ing each one
separately:
$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci main.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: main.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
SPECIAL NAMES: adding _cleanup_free_ as a attribute
$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci
logcontrol-example.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: logcontrol-example.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
But when you try to spatch both of them at once, Coccinelle starts
complaining and skipping the "bad" code:
$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci main.c logcontrol-example.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: main.c logcontrol-example.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
SPECIAL NAMES: adding _cleanup_free_ as a attribute
remapping: _cleanup_ to an ident in macro name
ERROR-RECOV: found sync end of #define, line 44
parsing pass2: try again
ERROR-RECOV: found sync end of #define, line 44
parse error
= File "logcontrol-example.c", line 44, column 21, charpos = 1719
around = '__attribute__',
whole content = #define _cleanup_(f) __attribute__((cleanup(f)))
badcount: 2
bad: #include <systemd/sd-journal.h>
bad:
BAD:!!!!! #define _cleanup_(f) __attribute__((cleanup(f)))
This was, unfortunately, hidden as it is visible only with
--verbose-parsing (or --parse-error-msg).
Another issue was how we handled includes. The original way of throwing
them into the pile of source files doesn't really work, leading up to
similar issues as above. The better way is to let Coccinelle properly
resolve all includes by telling it where to find our own include files
(basically the same thing we already do during compilation).
After fixing all this, Coccinelle now has a chance to process much more
of our code (there are still some issues in more complex macros, but
that requires further investigation). However, there's a huge downside
from all of this - doing a _proper_ code analysis is surprisingly time
and resource heavy; meaning that processing just one Coccinelle rule now
takes 15 - 30 minutes.
To make this slightly less painful, Coccinelle supports caching the
generated ASTs, which actually helps a lot - it gets the runtime of one
rule from 15 - 30 minutes down to ~1 minute. It, of course, has its own
downside - the cache is _really_ big (ATTOW the cache takes ~15 GiB).
However, even with the aggressive AST caching you're still looking at
~1 hour for one full Coccinelle run, which is a bit annoying, but I
guess that's the price of doing things _properly_ (but I'll definitely
look into ways of further optimizing this).
As the result is a bit funky (but still valid), i.e.:
static inline void iovec_done_erase(struct iovec *iovec) {
assert(iovec);
- iovec->iov_base = erase_and_free(iovec->iov_base);
- iovec->iov_len = 0;
+ *iovec = IOVEC_MAKE(erase_and_free(iovec->iov_base), 0);
}
Since the empty_to_null() function was "macrofied", we need to use a bit
of black magic to make Coccinelle avoid running the transformation on
the macro itself.
Follow-up to ef2409cbde.
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
RUN_WITH_UMASK was initially conceived for spawning externals progs with the
umask set. But nowadays we use it various syscalls and stuff that doesn't "run"
anything, so the "RUN_" prefix has outlived its usefulness.
This was a trivial wrapper that didn't provide any added value. With more
complicated structures like strvs, hashmaps, sets, and arrays, it is possible
to have an empty container. But in case of a list, the list is empty only when
the head is missing.
Also, we generally want the positive condition, so we replace many
if (!LIST_IS_EMPTY(x)) with just if (x).
Yu Watanabe
Daan De Meyer
Michael Vogt
David Tardon
Frantisek Sumsal
Jelle van der Waa
Lennart Poettering
Daan De Meyer
Matteo Croce
AtariDreams
Mike Yuan
Zbigniew Jędrzejewski-Szmek