-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
Opening a verity device is an expensive operation. The kernelspace operations
are mostly sequential with a global lock held regardless of which device
is being opened. In userspace jumps in and out of multiple libraries are
required. When signatures are used, there's the additional cryptographic
checks.
We know when two devices are identical: they have the same root hash.
If libcrypsetup returns EEXIST, double check that the hashes are really
the same, and that either both or none have a signature, and if everything
matches simply remount the already open device. The kernel will do
reference counting for us.
In order to quickly and reliably discover if a device is already open,
change the node naming scheme from '/dev/mapper/major:minor-verity' to
'/dev/mapper/$roothash-verity'.
Unfortunately libdevmapper is not 100% reliable, so in some case it
will say that the device already exists and it is active, but in
reality it is not usable. Fallback to an individually-activated
unique device name in those cases for robustness.
With gcc-9.1.1-2.fc31.x86_64 and -Doptimization=2:
../src/shared/dm-util.c: In function ‘dm_deferred_remove’:
../src/shared/dm-util.c:35:9: warning: ‘strncpy’ specified bound 128 equals destination size [-Wstringop-truncation]
35 | strncpy(dm.name, name, sizeof(dm.name));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
gcc is plain wrong here, because we checked strlen(name) a few lines above, so
there can be no truncation and even the terminator always fits. But let's avoid
the warning.
Lennart Poettering
Zbigniew Jędrzejewski-Szmek
Yu Watanabe
Luca Boccassi