This allows using it with CLEANUP_ARRAY(). For the 2 call sites
where we don't need to free the array, we do a regular for loop
calling json_variant_unref() instead.
When closing the FILE handle attached to a memstream, it may attempt to
do a realloc() that may fail during OOM situations, in which case we are
left with the buffer pointer pointing to NULL and buffer size > 0. For
example:
```
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
void *realloc(void *ptr, size_t size) {
return NULL;
}
int main(int argc, char *argv[])
{
FILE *f;
char *buf;
size_t sz = 0;
f = open_memstream(&buf, &sz);
if (!f)
return -ENOMEM;
fputs("Hello", f);
fflush(f);
printf("buf: 0x%lx, sz: %lu, errno: %d\n",
(unsigned long) buf, sz, errno);
fclose(f);
printf("buf: 0x%lx, sz: %lu, errno: %d\n",
(unsigned long) buf, sz, errno);
return 0;
}
```
```
$ gcc -o main main.c
$ ./main
buf: 0x74d4a0, sz: 5, errno: 0
buf: 0x0, sz: 5, errno: 0
```
This might do unexpected things if the underlying code expects a valid
pointer to the memstream buffer after closing the handle.
Found by Nallocfuzz.
This just adds the base functionality and some unit tests.
With TABLE_TIMESTAMP_RELATIVE we print "5s ago" and "5s left",
with the new TABLE_TIMESTAMP_LEFT, we print "5s ago" but "5s".
This is more useful for cases where we generally only want to
print timestamps in the future.
This rework the logic for handling the "header" cells a bit. Instead of
special casing the first row in regards to uppercasing/coloring let's
just intrduce a proper cell type TABLE_HEADER which is in most ways
identical to TABLE_STRING except that it defaults to uppercase output
and underlined coloring.
This is mostly refactoring, but I think it makes a ton of sense as it
makes the first row less special and you could in fact insert
TABLE_HEADER (and in fact TABLE_FIELD) cells wherever you like and
something sensible would happen (i.e. a string cell is displayed with
a specific formatting).
Originally, the table formatting code was written to display a number of
records, one per line, and within each line multiple fields of the same
record. The first line contains the column names.
It was then started to be used in a "vertical" mode however,
i.e. with field names on the left instead of the top. Let's support such
a mode explicitly, so that we can provide systematic styling, and can
properly convert this mode to JSON.
A new constructor "table_new_vertical()" is added creating such
"vertical" tables. Internally, this is a table with two columns: "key"
and "value". When outputting this as JSON we'll output a single JSON
object, with key/value as fields. (Which is different from the
traditional output where we'd use the first line as JSON field names,
and output an array of objects).
A new cell type TABLE_FIELD is added for specifically marking the
"field" cells, i.e. the cells in the first column. We'll automatically
suffic ":" to these fields on output.
All users were setting this to some static string (usually "-"), so let's
simplify things by not doing strdup, but instead limiting callers to a fixed
set of values. In preparation for the next commit, the function is renamed from
"empty" to "replacement", because it'll be used for more than empty fields. I
didn't do the whole string-table setup, because it's all used internally in one
file and this way we can immediately assert if an invalid value is passed in.
Some callers were (void)ing the error, others were ignoring it, and others
propagating. It's nicer to remove the boilerplate.
In the context of a table, both would be generally understood to have the same
meaning. "n/a" is a strange beast. It was useful when tables were produced on
the typewriter with "---------" used to separate rows. It is visually more
pleasing to use "-", and there is no risk of it being mistaken for a row
separator.
We expose various other forms of UUID helpers already, i.e.
SD_ID128_UUID_FORMAT_STR and SD_ID128_MAKE_UUID_STR(), and we parse
UUIDs, hence add a high-level helper for formatting UUIDs too.
This doesn't add any new code, it just moves some helpers
id128-util.[ch] → sd-id128.[ch], to make them public.
These functions are used pretty much independently of locale, i.e. the
only info relevant is whether th locale is UTF-8 or not. Hence let's
give this its own pair of .c/.h files.
Also,
- drop unnecessary +1 from buffer size, as IF_NAMESIZE or IFNAMSIZ
includes the nul at the end.
- format_ifname() does not update buffer on failure,
- introduces format_ifname_alloc(), FORMAT_IFNAME(), and their friends.
The new option --json= works with the 'security' verb and takes in one of three format flags.
These are off which is the default, pretty and short which use JSON format flags for output.
When set to true, it generates a JSON formatted output of the security analysis table. The
format is a JSON array with objects containing the following fields: set which indicates if
the id has been set or not, name which is what is used to refer to the id, json_field
which is the equivalent JSON formatted id name only used for JSON outputs, description which
is an outline of the id state, and exposure which is an unsigned integer in the range 0.0..10.0,
where a higher value corresponds to a higher security threat. The JSON version of the table is
printed on the standard output file.
Example Run:
The unit file testfile.service was created to test the --json= option
maanya-goenka@debian:~/systemd (json-security)$ cat <<EOF >testfile.service
> [Service]
> ExecStart = echo hello
> PrivateNetwork = yes
> PrivateMounts = yes
> PrivateDevices = yes
> EOF
Both the JSON output and the security analysis table below have been truncated to increase readability.
1. Testing for when --json=off
maanya-goenka@debian:~/systemd (json-security)$ sudo build/systemd-analyze security --json=off --root= --offline=true
testfile.service --no-pager
/usr/lib/systemd/system/plymouth-start.service:15: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's
process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'.
Support for KillMode=none is deprecated and will eventually be removed.
/usr/lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating
/var/run/dbus/system_bus_socket → /run/dbus/system_bus_socket; please update the unit file accordingly.
/usr/lib/systemd/system/gdm.service:30: Standard output type syslog is obsolete, automatically updating to journal. Please update your
unit file, and consider removing the setting altogether.
/home/maanya-goenka/systemd/foo.service:2: Unknown key name 'foo' in section 'Unit', ignoring.
NAME DESCRIPTION EXPOSURE
✓ PrivateNetwork= Service has no access to the host's network
✗ User=/DynamicUser= Service runs as root user 0.4
✗ CapabilityBoundingSet=~CAP_SET(UID|GID|PCAP) Service may change UID/GID identities/capabilities 0.3
✗ CapabilityBoundingSet=~CAP_NET_ADMIN Service has administrator privileges 0.3
→ Overall exposure level for testfile.service: 8.3 EXPOSED 🙁
2. Testing for when --json=pretty
maanya-goenka@debian:~/systemd (json-security)$ sudo build/systemd-analyze security --json=pretty --root= --offline=true
testfile.service
/usr/lib/systemd/system/plymouth-start.service:15: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's
process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'.
Support for KillMode=none is deprecated and will eventually be removed.
/usr/lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating
/var/run/dbus/system_bus_socket → /run/dbus/system_bus_socket; please update the unit file accordingly.
/usr/lib/systemd/system/gdm.service:30: Standard output type syslog is obsolete, automatically updating to journal. Please update your
unit file, and consider removing the setting altogether.
/home/maanya-goenka/systemd/foo.service:2: Unknown key name 'foo' in section 'Unit', ignoring.
[
{
"set" : true,
"name" : "PrivateNetwork=",
"json-field" : "PrivateNetwork",
"description" : "Service has no access to the host's network",
"exposure" : null
},
{
"set" : false,
"name" : "User=/DynamicUser=",
"json-field" : "UserOrDynamicUser",
"decsription" : "Service runs as root user",
"exposure" : "0.4"
},
{
"set" : false,
"name" : "CapabilityBoundingSet=~CAP_SET(UID|GID|PCAP)",
"json_field" : "CapabilityBoundingSet_CAP_SET_UID_GID_PCAP",
"description" : "Service may change UID/GID identities/capabilities",
"exposure" : "0.3"
},
{
"set" : false,
"name" : "CapabilityBoundingSet=~CAP_NET_ADMIN",
"json_field" : "CapabilityBoundingSet_CAP_NET_ADMIN",
"description" : "Service has administrator privileges",
"exposure" : "0.3"
},
...
]
3. Testing for when --json=short
maanya-goenka@debian:~/systemd (json-security)$ sudo build/systemd-analyze security --json=short --root= --offline=true
testfile.service
/usr/lib/systemd/system/plymouth-start.service:15: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's
process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'.
Support for KillMode=none is deprecated and will eventually be removed.
/usr/lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating
/var/run/dbus/system_bus_socket → /run/dbus/system_bus_socket; please update the unit file accordingly.
/usr/lib/systemd/system/gdm.service:30: Standard output type syslog is obsolete, automatically updating to journal. Please update your
unit file, and consider removing the setting altogether.
/home/maanya-goenka/systemd/foo.service:2: Unknown key name 'foo' in section 'Unit', ignoring.
[{"set":true,"name":"PrivateNetwork=", "json_field":"PrivateNetwork", "description":"Service has no access to the host's network","exposure":null}, ...]
In some cases it's useful to explicitly generate the JSON field names to
generate for table columns, instead of auto-mangling them from table
header names that are intended for human consumption.
This adds the infra and a test for it.
It's intended to be used by #20544, for the first column, which in text
mode should have an empty header field, but have an explicit name in
json output mode.
In general we almost never hit those asserts in production code, so users see
them very rarely, if ever. But either way, we just need something that users
can pass to the developers.
We have quite a few of those asserts, and some have fairly nice messages, but
many are like "WTF?" or "???" or "unexpected something". The error that is
printed includes the file location, and function name. In almost all functions
there's at most one assert, so the function name alone is enough to identify
the failure for a developer. So we don't get much extra from the message, and
we might just as well drop them.
Dropping them makes our code a tiny bit smaller, and most importantly, improves
development experience by making it easy to insert such an assert in the code
without thinking how to phrase the argument.
We recently started making more use of malloc_usable_size() and rely on
it (see the string_erase() story). Given that we don't really support
sytems where malloc_usable_size() cannot be trusted beyond statistics
anyway, let's go fully in and rework GREEDY_REALLOC() on top of it:
instead of passing around and maintaining the currenly allocated size
everywhere, let's just derive it automatically from
malloc_usable_size().
I am mostly after this for the simplicity this brings. It also brings
minor efficiency improvements I guess, but things become so much nicer
to look at if we can avoid these allocation size variables everywhere.
Note that the malloc_usable_size() man page says relying on it wasn't
"good programming practice", but I think it does this for reasons that
don't apply here: the greedy realloc logic specifically doesn't rely on
the returned extra size, beyond the fact that it is equal or larger than
what was requested.
(This commit was supposed to be a quick patch btw, but apparently we use
the greedy realloc stuff quite a bit across the codebase, so this ends
up touching *a*lot* of code.)
With some versions of the compiler, the _cleanup_ attr makes it think
the variable might be freed/closed when uninitialized, even though it
cannot happen. The added cost is small enough to be worth the benefit,
and optimized builds will help reduce it even further.
gcc 9.3.0 "cc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0" with --optimization=1 was
not able to figure out that all cases are covered because r is either set in
the switch or type < _TABLE_DATA_TYPE_MAX.
But for a human reader this might also not be obvious: the cases are not in
exactly the same order as enum definitions, and it's a long list. By using the
goto, there should be no doubt, and we avoid checking the condition a second
time.
Yu Watanabe
Mike Yuan
Lennart Poettering
Daan De Meyer
Frantisek Sumsal
Zbigniew Jędrzejewski-Szmek
Maanya Goenka
Luca Boccassi