Commit Graph
6034 Commits
Author SHA1 Message Date
Zbigniew Jędrzejewski-Szmek 337712e777 shared/json: stop using long double
It seems that the implementation of long double on ppc64el doesn't really work:
long double cast to integer and back compares as unequal to itself. Strangely,
this effect happens without optimization and both with gcc and clang, so it
seems to be an effect of how long double is implemented by the architecture.

Dumping the values shows the following pattern:
00 00 00 00 00 00 24 40  00 00 00 00 00 00 00 00   # long double v = 10;
00 00 00 00 00 00 24 40  00 00 00 00 00 00 80 39   # (long double)(intmax_t) v

Instead of trying to make this work, I think it's most reasonable to switch to
normal doubles. Notably, we had no tests for floating point behaviour. The
first test we added (for values even not in the range outside of double),
showed failures.

Common implementations of JSON (in particular JavaScript) use 64 bit double.
If we stick to this, users are likely to be happy when they exchange data with
those tools. Exporting values that cannot be represented in other tools would
just cause interop problems.

I don't think the extra precision would be much used. Long double seems to make
most sense as a transient format used in calculations to get extra precision in
operations, and not a storage or exchange format. So I expect low-level
numerical routines that have to know about hardware to make use of it, but it
shouldn't be used by our (higher-level) system library. In particular, we would
have to add tests for implementations conforming to IEEE 754, and those that
don't conform, and account for various implementation differences. It just
doesn't seem worth the effort.

https://en.wikipedia.org/wiki/Long_double#Implementations shows that the
situation is "complicated":

> On the x86 architecture, most C compilers implement long double as the 80-bit
> extended precision type supported by x86 hardware. An exception is Microsoft
> Visual C++ for x86, which makes long double a synonym for double. The Intel
> C++ compiler on Microsoft Windows supports extended precision, but requires
> the /Qlong‑double switch for long double to correspond to the hardware's
> extended precision format.

> Compilers may also use long double for the IEEE 754 quadruple-precision
> binary floating-point format (binary128). This is the case on HP-UX,
> Solaris/SPARC, MIPS with the 64-bit or n32 ABI, 64-bit ARM (AArch64) (on
> operating systems using the standard AAPCS calling conventions, such as
> Linux), and z/OS with FLOAT(IEEE). Most implementations are in software, but
> some processors have hardware support.

> On some PowerPC and SPARCv9 machines, long double is implemented as a
> double-double arithmetic, where a long double value is regarded as the exact
> sum of two double-precision values, giving at least a 106-bit precision; with
> such a format, the long double type does not conform to the IEEE
> floating-point standard. Otherwise, long double is simply a synonym for
> double (double precision), e.g. on 32-bit ARM, 64-bit ARM (AArch64) (on
> Windows and macOS) and on 32-bit MIPS (old ABI, a.k.a. o32).

> With the GNU C Compiler, long double is 80-bit extended precision on x86
> processors regardless of the physical storage used for the type (which can be
> either 96 or 128 bits). On some other architectures, long double can be
> double-double (e.g. on PowerPC) or 128-bit quadruple precision (e.g. on
> SPARC). As of gcc 4.3, a quadruple precision is also supported on x86, but as
> the nonstandard type __float128 rather than long double.

> Although the x86 architecture, and specifically the x87 floating-point
> instructions on x86, supports 80-bit extended-precision operations, it is
> possible to configure the processor to automatically round operations to
> double (or even single) precision. Conversely, in extended-precision mode,
> extended precision may be used for intermediate compiler-generated
> calculations even when the final results are stored at a lower precision
> (i.e. FLT_EVAL_METHOD == 2). With gcc on Linux, 80-bit extended precision is
> the default; on several BSD operating systems (FreeBSD and OpenBSD),
> double-precision mode is the default, and long double operations are
> effectively reduced to double precision. (NetBSD 7.0 and later, however,
> defaults to 80-bit extended precision). However, it is possible to override
> this within an individual program via the FLDCW "floating-point load
> control-word" instruction. On x86_64, the BSDs default to 80-bit extended
> precision. Microsoft Windows with Visual C++ also sets the processor in
> double-precision mode by default, but this can again be overridden within an
> individual program (e.g. by the _controlfp_s function in Visual C++). The
> Intel C++ Compiler for x86, on the other hand, enables extended-precision
> mode by default. On IA-32 OS X, long double is 80-bit extended precision.

So, in short, the only thing that can be said is that nothing can be said. In
common scenarios, we are getting only a bit of extra precision (80 bits instead
of 64), but use space for padding. In other scenarios we are getting no extra
precision. And the variance in implementations is a big issue: we can expect
strange differences in behaviour between architectures, systems, compiler
versions, compilation options, and even the other things that the program is
doing.

Fixes #21390.
2021-11-18 01:34:31 +01:00
Lennart Poettering 96603ea076 tree-wide: port various places over to open_mkdir_at() 2021-11-17 21:48:29 +01:00
Luca Boccassi b52b5763e7 hwdb: voidify call to mkdir_parents_label
CID#1466060
2021-11-17 17:47:48 +00:00
Lennart Poettering b049b48c4b Merge pull request #21326 from poettering/mkdir-tweaks
various tweaks to mkdir code
2021-11-16 22:55:11 +01:00
Lennart Poettering 71bf7ba193 user-record: relax rules on diskSize user record field
Let's not refuse low or high disk sizes unnecessarily early. They disk
sizes are subject fs limits anyway, hence there's no point in adding
another limit.

Relaxing thhe rules here as the advantage that we can later allow
"homectl resize lennart 0" as a generic way to minimize disk space.
2021-11-16 17:18:07 +01:00
Lennart Poettering 61fa16c1ca resize-fs: add helper that checks if the specified fs can do online grow/shrink
There's only one that can do this (btrfs), but let's abstract that fact,
a bit in case the other file systems learn this too one day.
2021-11-16 17:18:07 +01:00
Lennart Poettering 35cd0ba516 shared: clean up mkdir.h/label.h situation
Previously the mkdir_label() family of calls was implemented in
src/shared/mkdir-label.c but its functions partly declared ins
src/shared/label.h and partly in src/basic/mkdir.h (!!). That's weird
(and wrong).

Let's clean this up, and add a proper mkdir-label.h matching the .c
file.
2021-11-16 17:03:28 +01:00
Lennart Poettering ed304a5d73 mkdir: drop mkdir_errno_wrapper(), use mkdirat_errno_wrapper() instead
Let's reduce our code duplication, and let's focus on using xyzat()
style APIs more, hence drop mkdir_errno_wrapper() and stick to
mkdirar_errno_wrapper() wherever we can, it's a true superset of
functionality after all.
2021-11-16 17:03:03 +01:00
Lennart Poettering 3f692e2ece tree-wide: don't use mkdir_errno_wrapper() without reason
Simple mkdir() is fine, too, no need to use the wrapper
2021-11-16 17:02:58 +01:00
Lennart Poettering 3bb5ecaa88 mkdir-label: make mkdir_label() a wrapper around mkdirat_label() 2021-11-16 17:02:30 +01:00
Lennart Poettering fc0f4d6024 selinux: make mac_selinux_create_file_prepare() at wrapper around _at()
Let's make sure mac_selinux_create_file_prepare_at() works fine with
AT_FDCWD, and then make mac_selinux_create_file_prepare() just a inline
wrapper around it.
2021-11-16 17:02:03 +01:00
Lennart Poettering d1cd465e21 smack make mac_smack_fix_at() useful when called with dir_fd=AT_FDCWD 2021-11-16 17:01:58 +01:00
Lennart Poettering 67d0c6744f Merge pull request #21275 from keszybz/makefs-quiet
Makefs quiet output
2021-11-16 16:55:47 +01:00
Zbigniew Jędrzejewski-Szmek 2d96440fd5 makefs: add "support" for f2fs
The man page doesn't quite match what --help says, and I needed to use "-f" to
write a wiped partition. This all feels a bit experimental, but the fs has some
adherents, and we should make it easy to use.

(Also, an empty 256MB device formatted and mounted shows up as
  Filesystem      Size  Used Avail Use% Mounted on
  /dev/loop0      254M   85M  170M  34% /var/tmp/mount
which also seems a bit over the top…)

Requested in https://github.com/systemd/systemd/pull/21275#issuecomment-967928690.
2021-11-16 13:55:49 +01:00
Zbigniew Jędrzejewski-Szmek 8d433a99a4 makefs: fix too-long swap labels
Apparently mkswap has it's own limit, and it seems to be one lower than the one
for ext2/3/4.

$ for i in ext2 ext3 ext4 btrfs xfs vfat swap minix; do
  echo $i && wipefs -q -a '/var/tmp/głąbźśńćąśððð.img'
  build/systemd-makefs $i '/var/tmp/głąbźśńćąśððð.img'
done

ext2
/var/tmp/głąbźśńćąśððð.img successfully formatted as ext2 (label "głąbźśńćą", uuid 7626bc5c-8ac4-43cf-87b7-1b2761272dd3)
ext3
/var/tmp/głąbźśńćąśððð.img successfully formatted as ext3 (label "głąbźśńćą", uuid 0da22cad-0dbf-4a7a-962d-12cd39d006b5)
ext4
/var/tmp/głąbźśńćąśððð.img successfully formatted as ext4 (label "głąbźśńćą", uuid dded267b-8955-4d19-82a5-1f231d446059)
btrfs
/var/tmp/głąbźśńćąśððð.img successfully formatted as btrfs (label "głąbźśńćąśððð.img", uuid 9e2e89f1-010d-4ab6-80f3-f9e215dbc225)
xfs
/var/tmp/głąbźśńćąśððð.img successfully formatted as xfs (label "głąbźśń", uuid 2cc937af-4c41-465c-8f52-aab2304bd860)
vfat
mkfs.fat 4.2 (2021-01-31)
/var/tmp/głąbźśńćąśððð.img successfully formatted as vfat (label "G__B_______", uuid a3a9e028)

swap
...
LABEL=głąbźśńć, UUID=0ab787aa-37a6-4b32-978b-d71efc6e6098
/var/tmp/głąbźśńćąśððð.img successfully formatted as swap (label "głąbźśńć", uuid 0ab787aa-37a6-4b32-978b-d71efc6e6098)

minix
...
/var/tmp/głąbźśńćąśððð.img successfully formatted as minix (no label or uuid specified)
2021-11-16 13:55:48 +01:00
Zbigniew Jędrzejewski-Szmek 7ffe593b79 makefs: fix too-long ext2/3/4 labels 2021-11-16 13:55:46 +01:00
Zbigniew Jędrzejewski-Szmek dc91c971bf makefs: fix label for vfat filesystems
I was testing with a "test1.img" and mkfs.vfat rejects "TEST1.IMG" with the
error "Labels with characters *?.,;:/\|+=<>[]" are not allowed". So let's
replace those characters with "_".
2021-11-16 13:55:33 +01:00
Lennart Poettering 7c248223eb tree-wide: use new RET_NERRNO() helper at various places 2021-11-16 08:04:09 +01:00
Luca Boccassi e2f8c1124d Merge pull request #21277 from yuwata/permanent-hw-addr
network, udev: several improvement of permanent hardware address handling
2021-11-15 10:57:30 +00:00
Zbigniew Jędrzejewski-Szmek 4728625490 seccomp: move mprotect to @default
With glibc-2.34.9000-17.fc36.x86_64, dynamically programs newly fail in early
init with a restrictive syscall filter that does not include @system-service.
I think this is caused by 2dd87703d4386f2776c5b5f375a494c91d7f9fe4:

Author: Florian Weimer <fweimer@redhat.com>
Date:   Mon May 10 10:31:41 2021 +0200

    nptl: Move changing of stack permissions into ld.so

    All the stack lists are now in _rtld_global, so it is possible
    to change stack permissions directly from there, instead of
    calling into libpthread to do the change.

It seems that this call will now be very widely used, so let's just move it to
default to avoid too many failures.
2021-11-14 13:53:50 +09:00
Lennart Poettering 0fa90f7e04 uid-alloc-range: maintain only a single default alloc range structure
Either way we need these four values, let's simplify the code by keeping
only one const struct of this around.
2021-11-13 08:10:13 +01:00
Lennart Poettering b085d22406 shared: split out UID allocation range stuff from user-record.h
user-record.[ch] are about the UserRecord JSON stuff, and the UID
allocation range stuff (i.e. login.defs handling) is a very different
thing, and complex enough on its own, let's give it its own c/h files.

No code changes, just some splitting out of code.
2021-11-13 08:10:11 +01:00
Lennart Poettering 1ca5a6c76e Merge pull request #21333 from poettering/homed-report-fs-and-access-mode
homed: report actual home dir access mode and fs type in effect
2021-11-13 08:04:33 +01:00
Lennart Poettering b1beb00406 Merge pull request #21331 from poettering/luks-extra-mount-options
homed: allow per-user additional LUKS mount options
2021-11-13 08:03:55 +01:00
Jan Janssen 9cc6154608 test: Create convenience macros to declare tests 2021-11-13 09:20:24 +09:00
Yu Watanabe 1de88f3048 udev/net: read hardware address from netlink message 2021-11-13 08:53:33 +09:00
Yu Watanabe ed9fa69f1c ethtool-util: introduce ethtool_get_permanent_hw_addr()
And replaces all ethtool_get_permanent_macaddr() with it.
2021-11-13 08:53:33 +09:00
Lennart Poettering b0a7fb152a homed: include actual fs type + access mode as part of "status" section of user record
So far we have two properties for the intended fstype + access mode of
home dirs, but they might differ from what is actually used (because the
user record changed from the home dir, after it was created, or vice
versa). Let's hence add these props also to the "status" section of user
record, which report the status quo. That way we can always show the
correct, current settings.
2021-11-12 22:44:54 +01:00
Lennart Poettering 2e0001c281 homework: also add a way to configure additional mount options via a JSON user record field
Fixes: #15120
2021-11-12 22:22:06 +01:00
Lennart Poettering cbae575e0f keyring-util: add new keyring-util.h helpers
This adds to new helpers: keyring_read() for reading a key data from a
keyring entry, and TAKE_KEY_SERIAL which is what TAKE_FD is for fds, but
for key_serial_t.

The former is immediately used by ask-password-api.c
2021-11-12 22:15:06 +01:00
Lennart Poettering 52f05ef21d umask-util: add helper that resets umask until end of current code block 2021-11-12 16:01:40 +01:00
Zbigniew Jędrzejewski-Szmek 4f05a11c55 makefs: supress mkfs output, but print one line on success
$ for i in ext2 ext3 ext4 btrfs xfs vfat swap minix; do
  echo $i && wipefs -q -a /var/tmp/test2_img && build/systemd-makefs $i /var/tmp/test2_img
done

ext2
/var/tmp/test2_img successfully formatted as ext2 (label "test2_img", uuid ad584a5b-037b-497a-825d-eaf2ba90da2d)
ext3
/var/tmp/test2_img successfully formatted as ext3 (label "test2_img", uuid 95239fff-55f4-44d5-bae0-11ef75d13166)
ext4
/var/tmp/test2_img successfully formatted as ext4 (label "test2_img", uuid 8c7ea699-05ab-4ce6-8df6-bc20d53dfd29)
btrfs
/var/tmp/test2_img successfully formatted as btrfs (label "test2_img", uuid 860bb061-4d92-4607-8821-a9d00216490e)
xfs
/var/tmp/test2_img successfully formatted as xfs (label "test2_img", uuid f32499ea-7311-47bb-be57-da62e51d33ae)
vfat
mkfs.fat 4.2 (2021-01-31)
/var/tmp/test2_img successfully formatted as vfat (label "TEST2_IMG", uuid d1e4ae63)
swap
mkswap: /var/tmp/test2_img: insecure permissions 0644, fix with: chmod 0600 /var/tmp/test2_img

mkswap: /var/tmp/test2_img contains holes or other unsupported extents.
        This swap file can be rejected by kernel on swap activation!
        Use --verbose for more details.

Setting up swapspace version 1, size = 256 MiB (268431360 bytes)
LABEL=test2_img, UUID=16bc3d8c-98d4-462b-8ff8-338467cde871
/var/tmp/test2_img successfully formatted as swap (no label or uuid specified)
minix
21856 inodes
65535 blocks
Firstdatazone=696 (696)
Zonesize=1024
Maxsize=268966912

/var/tmp/test2_img successfully formatted as minix (no label or uuid specified)
2021-11-12 15:51:24 +01:00
Lennart Poettering c2caeb5d54 bootspec: catch up with sd-boot's bootspec implementation
Let's parse the same fields and use them the same way as in sd-boot.

Fixes: #20093
2021-11-11 17:22:59 +01:00
Lennart Poettering 31f7765762 bootspec: fix comment that says exactly the opposite of what is true 2021-11-11 17:22:27 +01:00
Luca Boccassi a704137c20 Merge pull request #21273 from yuwata/hostname-device-tree
hostnamed: use /proc/device-tree to get chassis type
2021-11-10 23:15:20 +00:00
Yu Watanabe 9133c1877f netif-util: fix stack-use-after-scope
Fixes a bug introduced by 0295b2fd1d.

Fixes #21292.
2021-11-10 22:11:18 +00:00
Yu Watanabe c6dce5720c condition: use /proc/device-tree/ 2021-11-11 04:42:50 +09:00
Lennart Poettering 04ba1bb038 shared: nudge people into sending us patches to make /lib64/ symlink generation work on all archs that need it
This is an attempt to nudge people into sending us patches that fix
bug #14311 for us, given that the original submitter lost interest.
2021-11-10 15:18:01 +01:00
Lennart Poettering d42e4fa258 shared: start pushing people gently to define GPT partition type UUIDs for missing archs
Let's generate a single gcc `#warning`  message asking people to define
partition type UUIDs for their architectures if they are missing.
2021-11-10 15:18:01 +01:00
Yu Watanabe 5ff3c71af8 Merge pull request #21276 from yuwata/ether-addr-util
ether-addr-util: introduce parse_hw_addr() and related conf parsers
2021-11-10 09:22:01 +09:00
Yu Watanabe 99628f363b conf-parser: introduce config_parse_hw_addr() and config_parse_hw_addrs() 2021-11-09 21:39:09 +09:00
Yu Watanabe c6df73ca72 ether-addr-util: introduce {hw,ether}_addr_hash_ops_free 2021-11-09 21:39:09 +09:00
Yu Watanabe aa4f765326 conf-parser: rename config_parse_hwaddr() -> config_parse_ether_addr() 2021-11-09 21:39:09 +09:00
Yu Watanabe 227e9ce255 ether-addr-util: replace ether_addr_from_string() with parse_ether_addr() 2021-11-09 21:39:09 +09:00
Lennart Poettering 4f538d7b22 tree-wide: use sd_event_source_disable_unref() where we can 2021-11-09 12:52:53 +01:00
Zbigniew Jędrzejewski-Szmek 8e93a614e7 makefs: also set uuid/label for ext2/ext3
We were only "supporting" ext4. Let's add "support", i.e. pass our optimization
options and uuid/label for the other two fses in the same family.

Nowadays there is separate code in the kernel, all three fs types are handled
by ext4 code. ext2 in particular is useful for in-memory devices: the journal
is just a waste of space there. ext3 is added for completeness mostly, since
ext4 should probably be used instead. But people might use it for testing or
for compatibility with older systems and I don't see much reason to not add
"support" here.
2021-11-09 09:12:01 +01:00
Zbigniew Jędrzejewski-Szmek 44cdeb6e33 makefs: reindent mkfs calls to follow the same style
I think this also makes it much easier to see what args are passed.
2021-11-09 09:12:01 +01:00
Yu Watanabe 0295b2fd1d netif-util: split net_get_unique_predictable_data() into two 2021-11-09 08:24:10 +09:00
Yu Watanabe 043b193f6e netif-util: rename net_get_name_persistent() -> net_get_persistent_name() 2021-11-09 08:24:10 +09:00
Yu Watanabe 4eef6fad31 netif-util: rename link_get_type_string() -> net_get_type_string() 2021-11-09 08:24:10 +09:00