Rootless Containers: Least Privilege for the Paranoid

There is a sentence that ought to be printed on the box every container runtime ships in: root inside the container is root on the host. Not metaphorically. Not “sort of”. By default, UID 0 in the container is UID 0 on the machine, and the only thing standing between those two facts is a set of kernel namespaces that a good enough bug will walk straight through.

I run a small homelab — a mix of services on a couple of boxes behind a reverse proxy — and for years I treated containers as a security boundary. They are a boundary, but a leaky one, and the leak all points the same way: outwards, onto my host, as root. Once I internalised that, my whole approach changed. I stopped asking “is this container safe?” and started asking “when this container gets popped, how boring can I make the aftermath?” That question has a name. It is least privilege, and this post is how I apply it to containers I actually run.

If you already avoid the Docker daemon on principle, some of this will be familiar — I made that argument in Podman for people who distrust the Docker daemon — but the hardening here applies whichever runtime you use.

The threat model, said plainly

Assume the workload will be compromised. Your code might be spotless and it happens anyway, because some dependency five layers down has a remote-code-execution bug you have never heard of, and one Tuesday it gets exploited. The attacker now has a shell inside your container. The interesting question is what that shell is worth.

A container is a normal Linux process with some namespaces (PID, network, mount, user, and friends) and some cgroup limits wrapped around it. Escapes happen. A kernel vulnerability in a syscall the container can reach, a bind-mount that exposes too much of the host, a Docker socket carelessly mounted into the container so the app can “manage its own containers” — any of these turns namespace isolation into a speed bump. And when the escape lands, the escaped process runs with whatever privileges the container had. If that was root, the attacker owns the host.

Least privilege means shrinking every one of those privileges in advance so that the escape, when it comes, lands in a padded cell. The container process should be an unprivileged nobody the kernel treats with contempt. Getting there is a stack of independent controls, and the beauty is that they compose — each one is worth having on its own, and together they are genuinely hard to get past.

User namespaces: making root a lie

The single highest-leverage control is the user namespace, because it attacks the root-equals-root problem at the root (sorry).

A user namespace lets a process see itself as UID 0 while the kernel, outside the namespace, sees it as some boring high-numbered UID with no privileges over anything. The mapping is configured through /etc/subuid and /etc/subgid, which hand each user a range of subordinate IDs. A typical line looks like this:

1

smarc:100000:65536

That grants my user 65,536 subordinate UIDs starting at 100000. Container UID 0 maps to host UID 100000, container UID 1 to 100001, and so on. So a process running as “root” inside the container is host UID 100000 — a user that owns nothing, can signal nothing, and has no capabilities over the host. If it escapes, it escapes into a body with no muscles.

Rootless Podman does this by default. You run podman run as your unprivileged user, and the container’s root is mapped into your subuid range automatically. This is the whole reason I moved most of my homelab workloads to rootless Podman: the safe thing is the default thing, and I don’t have to remember to configure it per container.

Docker can do the same via userns-remap, but it is opt-in and applies daemon-wide. You enable it in /etc/docker/daemon.json:

1
2
3

{
  "userns-remap": "default"
}

After a daemon restart Docker creates a dockremap user, reads its subuid/subgid ranges, and remaps every container into them. The catch is that it is all-or-nothing for the daemon, and it interacts awkwardly with some pre-existing named volumes (their files were written by real root and now the remapped root can’t read them). Worth it, but plan for the migration.

Belt and braces: don’t be root inside either

The user namespace makes container root harmless on the host. That does not mean you should run as root inside the container. Defence in depth wants both, because the two protect against different failures. The userns map protects the host if isolation breaks; running as an unprivileged internal user protects everything inside the container’s world — other mounted volumes, the app’s own files, the blast radius of an in-container exploit — and it protects you on the day someone runs the same image without a userns map by mistake.

So bake a non-root user into the image with USER in the Dockerfile, or override it at runtime with --user 10001:10001. Most application images have no business running as root; they need to read some config, bind a port, and write to a couple of directories. Give them an account that can do exactly that and nothing else. Reducing what lives inside the image in the first place helps here too — I go into that in Distroless and multi-stage builds: smaller, safer images, and a distroless image with a numeric USER pairs beautifully with everything in this post.

The rest of the stack: capabilities, privileges, syscalls, filesystem

Root on Linux was never one thing; it is a bag of capabilities. CAP_NET_BIND_SERVICE lets you bind low ports. CAP_SYS_ADMIN — the capability that famously means “basically root” — lets you mount filesystems and do a hundred other dangerous things. Containers start with a default subset of these capabilities granted, and the honest truth is that most services need none of them.

So drop the lot and add back only what the workload genuinely uses:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

podman run \
  --name webapp \
  --user 10001:10001 \
  --cap-drop=ALL \
  --cap-add=NET_BIND_SERVICE \
  --security-opt=no-new-privileges \
  --security-opt=seccomp=/usr/share/containers/seccomp.json \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --tmpfs /run:rw,noexec,nosuid,size=16m \
  -v webapp-data:/var/lib/webapp:U \
  -p 8080:8080 \
  registry.mylab.local/webapp:1.4.2

Walking through the security-relevant flags:

• --cap-drop=ALL then --cap-add=NET_BIND_SERVICE — start from zero capabilities and grant back only the one thing this app needs, the ability to bind a privileged port. If the app binds a high port and sits behind a proxy, you can drop even that and add nothing back. A process with an empty capability set is a much less interesting thing to compromise.
• --security-opt=no-new-privileges — this sets the kernel’s no_new_privs bit, which stops any child process from gaining privileges through a setuid binary or file capabilities during execve. If some setuid helper is lurking in the image, this defangs it. There is no reason a well-behaved app container should ever need to escalate, so nail the door shut.
• --security-opt=seccomp=... — seccomp filters which syscalls the container may make. The runtime’s default profile already blocks the genuinely dangerous ones (things like keyctl, ptrace against other processes, mount operations, kernel module loading). Passing the default here is explicit documentation, and the important discipline is what you must not do: when an app throws a permissions error, the wrong reflex is --security-opt seccomp=unconfined to make the message disappear. That reopens the entire syscall surface to shut up one log line.
• --read-only — the root filesystem is mounted read-only, so a compromised process cannot drop a binary in /usr/bin, tamper with the app, or persist anything across a restart. Almost every app writes to only a handful of paths, so you carve those out explicitly with --tmpfs (ephemeral scratch that vanishes on stop, mounted noexec,nosuid) and named volumes for real data.
• -v webapp-data:/var/lib/webapp:U — a persistent volume for the data the app must keep, with the :U suffix that fixes the ownership problem I describe below.

Notice what is not in that command. There is no --privileged. --privileged is the flag that undoes the entire post: it grants all capabilities, disables seccomp, disables the default AppArmor/SELinux confinement, and exposes host devices. It exists for a few legitimate cases (running Docker-in-Docker, some hardware access) and is otherwise a loaded gun pointed at your host. If a tutorial tells you to add --privileged to make something work, treat that as a bug report against the tutorial, not a solution.

Troubleshooting: where rootless bites back

None of this is free. Here are the problems I actually hit, and how I fixed each without reaching for the sledgehammer.

Volume files owned by a bizarre high UID. You mount a host directory or a fresh volume, the container writes to it, and on the host the files are owned by UID 100000-something. This is the userns shift doing exactly what it promised — container root is host UID 100000. It is confusing the first time and harmless once you understand it. In Podman the clean fix is the :U volume suffix, which tells Podman to recursively chown the volume’s contents to match the UID the container will run as. You can also do it by hand with podman unshare chown -R 10001:10001 /path/to/volume, which runs the chown inside the user namespace so the numbers line up. On Kubernetes the equivalent lever is fsGroup in the pod security context. Whatever the tool, the goal is to make the volume’s ownership and the container’s effective UID agree.

A service that needs a low port. By default an unprivileged user cannot bind ports below 1024, and rootless containers inherit that. Three ways out, in order of how much I like them: bind a high port inside the container and put a reverse proxy in front (my default — the app listens on 8080, the proxy owns 443); grant --cap-add=NET_BIND_SERVICE and lower the threshold with sysctl net.ipv4.ip_unprivileged_port_start=80; or, least favourite, publish with -p 443:8443 and let the port mapping do the translation. The proxy approach keeps every app on a boring high port and centralises TLS, which is where I want it anyway.

An app that crashes under --read-only. Something inside insists on writing to a path you didn’t expect — a lock file in /var/run, a cache in /app/.cache, an SQLite journal next to the database. The temptation is to give up and drop --read-only. Don’t. Find the exact path from the error and mount a --tmpfs there (for scratch) or a named volume (for data you must keep). You end up with a read-only root and a short, explicit list of writable paths, which is a far stronger position than a wide-open filesystem. It takes two or three iterations to discover every path an app wants; that is normal, and the resulting config documents precisely what your app touches.

Rootless memory and CPU limits silently doing nothing. You set --memory=512m on a rootless container and the limit is ignored, because your user has not been delegated the relevant cgroup v2 controllers. On a systemd + cgroups v2 host, delegation is handled per-user; you enable the controllers for your user’s slice (a drop-in under /etc/systemd/system/[email protected]/ setting Delegate=cpu cpuset io memory pids), then re-login. Until the controllers are delegated, the runtime cannot enforce those limits and will often warn rather than error, so it is easy to miss. Check cat /sys/fs/cgroup/user.slice/.../cgroup.controllers to confirm memory and cpu are present.

The recurring temptation of --privileged. Every one of the problems above has a lazy shortcut that is really just --privileged (or --cap-add=ALL, or seccomp=unconfined) wearing a smaller hat. Each time you reach for one to silence an error, you are trading a five-minute fix for a permanent hole. I keep a rule for myself: if the fix widens privilege, I have to be able to say out loud which specific capability or syscall the app needs and why. If I can’t, I haven’t understood the problem yet, and disabling the guard is guessing with the safety off.

Where rootless genuinely can’t go

Honesty section, because the paranoid deserve the caveats. Rootless has real limits. Binding ports below 1024 needs the sysctl or a proxy, as above. Workloads that truly require host-level capabilities — certain VPN tools that manage the host’s network stack, some storage and clustering software that expects real CAP_SYS_ADMIN — will not run rootless without compromises, and for those I either accept a carefully-scoped rootful container or run them outside containers entirely, on a box I treat as expendable. Storage is the other historical wart: rootless overlay filesystems long meant fuse-overlayfs, which carries a performance penalty versus the kernel’s native overlay. Modern kernels support native rootless overlay and Podman uses it where it can, so on an up-to-date host this mostly sorts itself out, but on older kernels you will still see fuse-overlayfs and the I/O tax that comes with it.

The verdict

Rootless containers with a dropped capability set, no-new-privileges, the default seccomp profile, and a read-only root are the closest thing I have found to making a container compromise dull. The attacker gets a shell as a nobody, in a filesystem they can’t write to, with no capabilities, unable to make the syscalls that would let them escalate or escape, mapped to a host UID that owns nothing. That is the whole goal: assume you will be popped, and arrange for the popping to be worthless.

It costs you an afternoon of fiddling with UID mappings and writable-path lists per workload, and it costs you a handful of edge cases that rootless simply won’t serve. I think that is a bargain. My homelab runs a couple of dozen services this way, and the day one of them has a bad afternoon, I would very much like the blast radius to be a wiped container and a shrug rather than a host rebuild and a long, cold think about what else the attacker touched. Set it up once, per the command above, and make the paranoid version your default. Future you, reading an incident log that ends in “contained”, will be grateful.