Docker Socket Proxy: Closing the Biggest Hole in Your Stack
Your reverse proxy has root on the host. It only ever needed to read a list.
Contents
Go and read your compose files. Count the containers with this line in them:
| |
I found six. A reverse proxy, a container manager, an update notifier, a log viewer, a metrics exporter, and a dashboard that generates its own tiles from container labels. Six containers, each of which had complete, unrestricted, unauthenticated root on the host, because that is what the Docker socket is.
The :ro does nothing. It makes the socket file read-only, and a socket is an API you
talk to rather than a file whose contents you read. A process that can talk to it can
issue
POST /containers/create with Binds: ["/:/host"] and Privileged: true, and be root on
the host filesystem a second later. Read-only on the mount is theatre, and I put it there
for years because everyone else did.
Here is the fix, and an honest account of how much it actually fixes.
The threat, stated plainly
Nobody is going to break into your house and type docker run -v /:/host. The realistic
path is: one of those six containers has a vulnerability, someone reaches it, gets code
execution as whatever user that container runs as, and finds a socket sitting there. The
container was a log viewer with no business writing anything. It is now the machine.
This is a privilege escalation with a step count of one, and it converts “an information disclosure bug in a dashboard” into “total compromise of the host and everything on it”. That leap is the thing worth removing, and it maps directly onto the exercise in a homelab threat model: the question is never whether you will be attacked by a determined adversary, it is how far a boring, automated compromise of one exposed service gets to travel.
The insight that makes the fix cheap is that none of those six containers needs the API.
They need a fraction of it. My reverse proxy needs to list containers and read their
labels. The update notifier needs to list containers and read their image names. The log
viewer needs to list containers and read their logs. Not one of them needs
POST /containers/create, and none of them will ever miss it.
The proxy
tecnativa/docker-socket-proxy is HAProxy with an ACL file. It mounts the real socket,
listens on TCP port 2375, and permits or denies requests based on the API path, controlled
entirely by environment variables. Everything defaults to denied.
| |
Two details carry most of the weight.
internal: true on the network means Docker creates no gateway to the outside world. The
proxy has no route off the host, so even a compromised proxy has nowhere to send anything.
Only containers explicitly attached to this network can reach port 2375, and 2375 is
published nowhere.
POST: 0 is the load-bearing line. The Docker API’s destructive operations are all POST.
With POST denied, an attacker who reaches this proxy can enumerate — read container names,
labels, environment variables, network settings — and can do nothing at all. No create, no
exec, no start, no delete.
Then the consumer, with the socket mount removed and a DOCKER_HOST in its place:
| |
Traefik reads DOCKER_HOST natively and needs no other change. Restart it and everything
keeps working, because the label-reading it does was always a GET /containers/json.
Most tools follow the same convention. My update notifier — I wrote about
Diun recently, and it was one of
the six — takes the same variable and behaves identically. So does the log viewer. If a
tool has no DOCKER_HOST support it usually has a config key for the socket path that
accepts a TCP URL.
The per-tool inventory
The whole point is granting each consumer the least it can function with, so they get their own proxies. Three environment variables’ difference in behaviour:
| Consumer | Needs | Settings |
|---|---|---|
| Reverse proxy | container list + labels | CONTAINERS=1 |
| Update notifier | container list + image names | CONTAINERS=1 |
| Log viewer | container list + logs | CONTAINERS=1 |
| Metrics exporter | containers, and often info | CONTAINERS=1 INFO=1 |
| Dashboard | container list | CONTAINERS=1 |
| Container manager | everything | nothing to be done |
The last row is the honest one. A tool whose job is to start and stop containers needs
POST=1, and POST=1 with CONTAINERS=1 is enough to create a privileged container. You
cannot proxy your way out of that — the tool’s function is the dangerous capability. The
proxy still helps a little, since a separate instance for that one tool limits what the
other five can do, and the manager itself remains a full-privilege service that should live
behind real authentication.
Five out of six is a good afternoon. Pretending the sixth is solved would be a lie.
What this does not fix
Three things, and the first one surprised me.
CONTAINERS=1 leaks your secrets. GET /containers/{id}/json includes the container’s
full environment. Every password, API key and token you passed as an environment variable
is readable by anything with container list access — which is now five of my six tools,
where before it was five of my six tools plus root. The exposure was always there; the
proxy simply made me look at it. The fix is orthogonal: pass secrets as files, use the secrets: key
in compose, and keep them out of the environment entirely.
The proxy is a path filter. It matches on API endpoints, and it has no idea which
container you are asking about. CONTAINERS=1 means all containers, so the log viewer
can read the database container’s logs and the reverse proxy can read everything’s labels.
There is no per-object scoping, because the Docker API has no per-object authorisation for
HAProxy to express. Docker’s authorisation plugin interface can do this properly and
nobody has written a homelab-friendly one.
It is another container. A small one — HAProxy, about 8 MB, no shell worth the name — but it holds the real socket, so compromising the proxy is compromising the host. You have gone from six containers with root to one container with root and five with read access. That is a real, large improvement, and it is a reduction rather than an elimination. The proxy is worth trusting more than a dashboard is because its attack surface is one HAProxy config and no application logic.
Doing the migration without a bad evening
Six containers, one at a time, over about a week. The order matters and the method is boring, which is how you want a change to your reverse proxy to feel.
Start by finding them all. This is the inventory command, and it will surprise you:
| |
Six. I would have guessed four, and the two I forgot were the dashboard and the metrics exporter — both installed on an idle Sunday, both entirely uninteresting, both root on the box for eighteen months.
Then, per container: stand up its socket proxy with everything at zero, point the consumer
at it, restart, and watch the proxy’s logs while you exercise the tool. Every 403 is a
line in the log naming the endpoint it wanted. Turn on that one thing, restart, repeat.
Three iterations is typical, and after the third the tool works and you have an exact
inventory of what it needs — which is a more interesting document than it sounds. My
dashboard, which renders tiles from container labels, wanted INFO, VERSION and
PING before it would draw anything, and none of those endpoints tell it anything it
displays. Software asks for what it was written to ask for.
Do the reverse proxy first, because it is the most exposed thing you run and its requirements are the smallest. Do the container manager last, or never — its POST access is its purpose, and the honest resolution there is to run it behind authentication and treat it as a privileged administrative service, in the same tier as an SSH session.
One shared proxy for all five read-only consumers works fine and is what most people do. I
run separate instances because they are 8 MB each and it lets the metrics exporter have
INFO=1 while nothing else does. That is fussy. It also means a compromise of one tool
does not inherit another tool’s grants, which is the entire idea, applied consistently.
Why the Docker API is like this
Worth a paragraph of history, because the design looks careless until you know where it came from.
The Docker daemon runs as root because containers need root-level operations: creating
namespaces, mounting filesystems, configuring cgroups, wiring network interfaces. In 2013
that was a reasonable architecture and there was no unprivileged alternative on Linux
worth using. The socket is the daemon’s control channel, and the daemon deliberately has no
authentication or authorisation on it — the model is that filesystem permissions on the
socket are the security boundary. If you can open /var/run/docker.sock, you are trusted
completely, because Docker assumed only root and the docker group could.
Everything downstream follows from that assumption. Adding a user to the docker group is
documented as a convenience so you can drop sudo, and it is precisely equivalent to
putting them in sudoers with NOPASSWD: ALL. Handing the socket to a container is the
same act, aimed at software instead of a person, and it became a convention because the
alternative — teaching every tool a real authentication protocol — was work nobody did.
Docker’s authorisation plugin interface exists to fix this properly. A plugin sees each API request with its full body and can approve or deny it on any criteria, including which container is being addressed. It has been there since 2016. In a decade I have found essentially nothing usable for a homelab built on it, because writing one means implementing a daemon and a policy language, and the people who need it most are the least equipped to write it. So we get HAProxy with an ACL list, which filters on the one thing visible in the URL — the endpoint — and calls it a day.
Understanding that shapes what you expect. Treat the proxy as the coarsest possible filter, sitting in a place where no filter existed at all. The reason it helps as much as it does is that the gap between “everything” and “two GET endpoints” stays enormous even when the filter is crude.
Troubleshooting
Traefik logs “Provider connection error, retrying”. The proxy denied something. HAProxy logs every decision:
| |
The 403 tells you exactly which endpoint to enable — here, VERSION=1. Work through the
logs and turn on the minimum that stops the errors. Resist enabling things speculatively.
Everything 403s including the container list. Check the API version prefix. Some
older tools request /v1.24/containers/json; the proxy handles versioned paths fine, so
if everything fails, you have more likely attached the consumer to the wrong network and
are getting a connection refused that the client is reporting oddly.
“Cannot connect to the Docker daemon at tcp://socket-proxy:2375”. DNS on a
user-defined network resolves by service name, so this is usually the consumer sitting on
a different network than the proxy. docker network inspect socket-proxy-net lists what is
actually attached.
It worked, then stopped after a compose recreate. The internal: true network is
recreated with a new subnet, and long-lived clients that cached the address do not
re-resolve. Restart the consumer.
A tool needs EXEC=1 for its health check. Some do — a monitoring agent running
docker exec to check a database. EXEC=1 combined with POST=1 is another route to
host root. Change the health check to something the container exposes over HTTP instead,
and if that is impossible, decide consciously that this tool is in the trusted tier.
Is it worth it?
An hour, one small container, and a DOCKER_HOST line per consumer. Against that, the
largest single privilege concentration in a typical homelab drops by most of its size. I
know of no other change with that ratio.
There is a better answer available, and it takes longer. Podman’s rootless API socket does
this properly at the identity layer — the socket belongs to an unprivileged user, and a
tool that reaches it can only act as that user, so POST /containers/create with the host
root mounted simply fails on permissions. That is a real fix at the identity layer, one level below where any filter operates, and
it is where I have been moving the machines that matter, as covered in
rootless Podman for the paranoid. It is also a
migration, and migrations take months of evenings.
The socket proxy is what you do on the Docker hosts you are not migrating this year. It is unglamorous, it takes an afternoon, and it turns five root-equivalent containers into five read-only ones. Start with the one facing the internet — for most people that is the reverse proxy, which is both the most exposed service you run and, wonderfully, the one that needs the least from the API. It is a ten-minute change on the container most likely to be attacked.
Do that one tonight. The other five can wait for the weekend.
Related Content
