A write through an overlay mount does not create the object on the overlay
itself: overlayfs creates it on the upper layer via vfs_create(), which
reaches only the *inode* LSM hooks, not the *path* hooks the transient
range policy attaches to. The one path hook that does fire sees the overlay
mount, whose superblock belongs to the (managed) mounting namespace and so
looks like it dies with it -- while the upper may be a persistent file
system on which the transient IDs then persist.
Regular file system -- caught by the path hook:
openat(dir_on_persistent_fs, "f", O_CREAT)
path_openat -> ... -> security_path_mknod(path{mnt = persistent mount}) << path hook (attached)
userns_restrict_path_mknod -> validate_mount()
sb->s_user_ns = init -> not below managed (outlives the ns)
mnt_idmap does not remap the transient id (reaches disk)
=> -EPERM BLOCKED
Overlay write -- slips past the path hook (this is the hole):
openat(overlay_mnt/"f", O_CREAT)
path_openat -> ... -> security_path_mknod(path{mnt = OVERLAY mount}) << path hook (attached)
validate_mount(): overlay sb->s_user_ns == managed
=> "dies with the namespace" ALLOW
ovl_create_or_link -> vfs_create(UPPER dentry)
security_inode_create(UPPER inode) << inode hook (NOT attached)
=> no check
inode created on the UPPER fs, owned by the transient uid PERSISTS
Judge the upper at mount time instead, in lsm/sb_kern_mount:
mount("overlay", ..., upperdir=<X>, workdir=<Y>)
do_new_mount -> vfs_get_tree
ovl_fill_super
stamps work/index onto the upper via vfs_mkdir, owned by the
transient mounter -> security_inode_mkdir only (not a path hook)
security_sb_kern_mount(sb) << hook (attached)
userns_restrict_sb_kern_mount:
sb->s_magic != OVERLAYFS_SUPER_MAGIC -> return 0 (not an overlay)
find_managed_userns(sb->s_user_ns) == none -> return 0 (unmanaged mounter)
upper = ((struct ovl_fs*)sb->s_fs_info)->layers[0].mnt
upper == NULL -> return 0 (read-only, no upper)
fs_userns = upper->mnt_sb->s_user_ns
upper dies with the managed ns (e.g. a tmpfs) => return 0 ALLOW
upper persistent, idmap records no transient id => return 0 ALLOW
upper persistent, an idmap extent records a transient => -EPERM REFUSE
The upper has to be judged at mount time because overlayfs performs the
actual upper creations (both the mount-time work/index dirs and every later
copy-up) with internal vfs_create()/vfs_mkdir() calls that reach only the
inode hooks, on the upper's own superblock, where neither the overlay's
ovl_fs nor the mount idmapping is available. Refusing the mount stops every
such write. It cannot undo the work/index dirs ovl_fill_super() already
stamped onto the upper before this hook runs. Reaching those takes an
upperdir and a workdir that already exist on the same persistent,
non-idmapped file system and that the transient IDs may write to: the client
cannot create them itself, since that is what the path hooks deny.
The idmap check walks the upper mount's uid_map and gid_map and permits the
overlay only when no extent records a transient id on disk (ids the idmap
does not map fail the write, so they are fine too). This is what lets a
client back a writable overlay with a persistent but id-mapped upper, e.g.
one handed out by mountfsd, while a plain persistent upper stays refused.
The check keys on whether a writable upper exists (layers[0].mnt is
non-NULL), which is fixed for the superblock's lifetime, not on the mutable
SB_RDONLY flag: an overlay mounted read-only but with an upper can be
remounted read-write later without re-entering sb_kern_mount (remount uses a
different LSM hook), so it must be judged now. A genuinely read-only overlay
has no upper and is left alone. Reading the upper layer needs overlayfs'
private struct ovl_fs/ovl_layer, which are absent from vmlinux.h when
overlayfs is built as a module; they are declared as CO-RE type flavors and
guarded by bpf_core_type_exists(), and the service now orders itself after
modprobe@overlay.service so the module's BTF is present by the time the
program is loaded.
A managed (nsresourced-provisioned) user namespace can therefore only mount
a writable overlay whose upper dies with it (e.g. a tmpfs it set up
itself). Callers that need a writable, persistent (non-tmpfs) upperdir --
mkosi, for one -- should use a regular unprivileged user namespace for that
rather than a transient nsresourced range, since such an overlay is refused
for a managed namespace by design.
test-userns-restrict now mounts real overlays: a read-only overlay, a
writable overlay with a tmpfs upper, and a writable overlay whose upper is
an init-owned but idmapped mount are all allowed, while a writable overlay
with a plain init-owned upper is refused with EPERM. The kernel permits that
last mount without the LSM loaded, so the EPERM is the policy's doing.
nsresourced hands out transient, recyclable UID/GID ranges to
unprivileged user namespaces. So that a client cannot gain persistence by
leaving inodes owned by its transient range on a file system that outlives
the namespace -- where a later, unrelated client that recycles the range
would inherit them -- a BPF-LSM policy vetoes inode creation and chown()
that would end up owned by a transient ID.
The policy this replaces keyed its decision on properties of the mount a
write targeted. Those can be forged: a single unshare(CLONE_NEWNS) hands
the namespace clones of every mount it can see, each backed by the very
same superblock but no longer recognisable by the original check, and the
mount namespace's owning user namespace is attacker-chosen as well.
Decide instead from two questions about the object the write would actually
create, both answered from fields the kernel stamps itself and that survive
mount cloning and cannot be forged with unshare():
a) Does the file system die with the user namespace? A superblock records
the user namespace it was mounted in (sb->s_user_ns); if that sits at
or below the managed namespace, nothing it carries can outlive the
transient range.
b) Will the transient ID actually reach the disk? An idmapped mount
(vfsmount->mnt_idmap) may translate it into something else first, in
which case nothing recyclable is recorded. This mirrors the kernel's
own from_vfsuid()/from_vfsgid().
Walking the user-namespace chain upwards, rather than looking only at the
exact namespace, is what keeps a nested namespace from shaking off the
policy.
The map schema changed incompatibly (the old mount-id allowlist became the
flat userns_managed inode set), so the BPF objects are pinned under a fresh
directory, /sys/fs/bpf/systemd/userns-restrict-v2, and the obsolete
unsuffixed tree from previous versions is removed on install. Removing the
old link pins detaches the previous, bypassable programs, so they do not
linger attached next to the new ones.
Whenever delegating UID ranges to a user namespace, it can also be
useful to map the foreign UID range, so that the container running in
the user namespace with delegated UID ranges can download container
images and unpack them to the foreign UID range.
Let's add an option mapForeign to make this possible. Note that this option
gives unprivileged users full access to the any foreign UID range owned directory
that they can access. Hence it is recommended (and already was recommended) to
store foreign UID range owned directories in a 0700 directory owned by the
owner of the tree to avoid access and modifications by other users.
This is already the case for the main users of the foreign UID range,
namely /var/lib/machines, /var/lib/portables and /home/<user> which all
use 0700 as their mode.
Users will also be able to create foreign UID range owned inodes in any
directories their own user can write to (on most systems this means /tmp,
/var/tmp and /home/<user>).
mkosi does all of its environment setup in an unprivileged user
namespace with an identity mapping. When it invokes nspawn and nspawn
tries to get a transient userns from nsresourced, this fails as no
transient ranges are mapped into mkosi's unprivileged userns (as doing
so would require privileges).
To fix this problem, let's allow allocating unprivileged self user
namespaces in nsresourced, similar to what the kernel allows, except that
we also support delegations for these. This means that mkosi can get its
unprivileged userns as before from nsresourced, but it can also request a
delegated 64K range inside that userns as well, which nsresourced can then
allocate to nspawn later when it asks for one.
Similar to the kernel, we disallow setgroups for self mappings. However,
instead of doing this via /proc/self/setgroups, which applies to the current
user namespace and all its child user namespaces, we use the BPF LSM to deny
setgroups instead, so that it can still be allowed for child user namespaces.
We need this because as soon as a container launches in a child user namespace
using one of the delegated mappings, it has to be able to do setgroups() to be
able to function properly.
To allow mapping the root user, we need to add the CAP_SETFCAP capability to
nsresourced.
We want to support the scenario where we bind mount the nsresourced
varlink socket into a container to allow nested containers where the
outer container runs in its own transient range from nsresourced but
can still allocate transient ranges for its own nested containers.
To support this use case let's add support for delegation. Delegated
ranges are allocated when allocating the primary range and are propagated
1:1 to the user namespace. We track delegated ranges in ".delegate" files
in the userns registry so that they can't be used for other range allocations.
We make one exception for delegated ranges though, if we get a request from
a user namespace that is a child of the user namespace that owns the delegated
ranges, we allow allocating from the delegated range. The parent userns already
has full ownership over the child userns, so it doesn't matter that the parent
userns and the child userns share the same range. This allows making use of
delegated ranges without having to run another copy of nsresourced inside the
parent userns to hand out from the delegated range.
To support recursive delegations, we keep track of the previous owners of the
delegated range and restore ownership to the last previous owner when the current
owner is freed.
This adds a small, socket-activated Varlink daemon that can delegate UID
ranges for user namespaces to clients asking for it.
The primary call is AllocateUserRange() where the user passes in an
uninitialized userns fd, which is then set up.
There are other calls that allow assigning a mount fd to a userns
allocated that way, to set up permissions for a cgroup subtree, and to
allocate a veth for such a user namespace.
Since the UID assignments are supposed to be transitive, i.e. not
permanent, care is taken to ensure that users cannot create inodes owned
by these UIDs, so that persistancy cannot be acquired. This is
implemented via a BPF-LSM module that ensures that any member of a
userns allocated that way cannot create files unless the mount it
operates on is owned by the userns itself, or is explicitly
allowelisted.
BPF LSM program with contributions from Alexei Starovoitov.
Luca Boccassi
Daan De Meyer
Christian Brauner
Daan De Meyer
Lennart Poettering
Yu Watanabe