Distroless Container Images: Smaller, Safer, and More Annoying to Debug
No shell, no package manager, no CVEs you didn't put there yourself
Contents
The first time you run a vulnerability scanner against a node:20 image and it reports two hundred CVEs, you have a small crisis of faith. None of those flaws are in your code. They’re in bash, curl, apt, coreutils, the Debian base — an entire operating system you shipped just to run one process. Distroless images are the reaction to that absurdity: a container image containing your application, its runtime dependencies, and nothing else. No shell. No package manager. No ls. The CVE count drops because the attack surface drops, and that’s not a coincidence — it’s the whole point.
What “distroless” actually means
Google’s distroless images are minimal base images that include only what a language runtime needs: glibc, CA certificates, /etc/passwd, timezone data, and for the language-specific variants, the runtime itself. There’s no /bin/sh. You cannot docker exec -it into one and poke around, because there is nothing to poke around with. That sounds hostile, and it is — deliberately. An attacker who lands a remote code execution in your app finds no shell to spawn, no wget to pull a second-stage payload, no package manager to install tools. They’re stuck in a near-empty box with one running process.
The natural way to use them is a multi-stage build: do all your messy compilation in a fat builder image, then copy only the artefact into the distroless final stage.
| |
That final image is a handful of megabytes, runs as non-root by default, and a scanner finds essentially nothing to complain about because there’s essentially nothing there. For a statically-linked Go or Rust binary you can go further still with gcr.io/distroless/static or even scratch.
What actually got safer, and what didn’t
It’s worth being precise about the security claim, because “fewer CVEs” can be a vanity metric. Most of those two hundred CVEs on a node:20 image are in packages your application never invokes — a flaw in apt does nothing if apt is never run. So distroless doesn’t make you “200 vulnerabilities safer” in any literal sense; a lot of those were unreachable to begin with. What it does is something more durable: it removes the post-exploitation toolkit.
The threat model that distroless genuinely improves is the one after the initial breach. An attacker who pops your app via a deserialisation bug or an SSRF wants to do three things: look around, pull down a second-stage payload, and establish persistence. In a fat image they have bash, curl, wget, python, a package manager, and a writable filesystem to do all of it. In a distroless image they have your single binary and a near-empty root. There’s no interpreter to run a downloaded script, no shell to pipe a reverse connection through, no chmod to make a dropped file executable. They’re not stopped — a determined attacker can still operate purely in-memory inside your process — but the bar is dramatically higher and the activity is far noisier. That’s the win that matters, and it’s why distroless pairs naturally with image-signing and scanning rather than replacing them. I still run Trivy across every image before it ships, because distroless shrinks the haystack but doesn’t guarantee there’s no needle — your own dependencies still carry their own CVEs, shell or no shell.
What distroless does not fix is the integrity of the image itself. A minimal image pulled from a registry you don’t control is still a thing you’re trusting blindly. Pinning by digest rather than tag, and verifying provenance with something like Sigstore and cosign, closes the gap distroless leaves open: a tiny image from an untrusted source is still an untrusted image.
The debugging tax is real
Here’s the part nobody puts on the marketing slide. The day your container is crash-looping in production and you reach for the usual reflexes:
| |
There is no shell. There never was. Your muscle memory is useless. You can’t cat a config file, can’t curl localhost:8080/health, can’t ps. Everything you’d normally do to inspect a misbehaving container is gone along with the attack surface you were so pleased to remove. This is the trade, stated honestly: you have made the image safer and you have made it much harder to inspect by hand.
The workarounds exist and you should know them before you need them, not during an incident.
The cleanest is an ephemeral debug container that shares the broken container’s namespaces. On Docker:
| |
On Kubernetes the equivalent is first-class:
| |
Both give you a shell next to the application — sharing its network and process view — without baking any of that into the shipped image. There’s also :debug tags of the distroless images themselves, which include a minimal BusyBox shell for non-production builds. Use those in staging if you must, never in prod.
The gotchas that bite during the build
Beyond the missing-shell shock, a predictable set of failures catches people the first time, and they’re worth pre-empting because the error messages are unhelpful.
The CA-certificate trap. Your binary makes an HTTPS call and gets x509: certificate signed by unknown authority, even though it worked fine in the builder. The fat builder image had the system CA bundle; the distroless static variant might not, depending on which you picked. The base and static distroless images do ship /etc/ssl/certs/ca-certificates.crt, but scratch ships nothing — so if you went all the way to scratch, copy the certs in explicitly:
| |
Timezone and user data. Code that calls time.LoadLocation("Europe/London") will fail without /usr/share/zoneinfo, and anything that looks up a username needs /etc/passwd. The distroless images include both; scratch does not. For Go specifically, building with -tags timetzdata bakes the timezone database into the binary and removes the dependency entirely — the kind of small detail that saves a confused hour.
CGO and dynamic linking. The single most common distroless failure is shipping a dynamically-linked binary into an image with no dynamic loader. The symptom is brutal: the container exits instantly with no log output at all, because the kernel can’t even start the process. The fix is the CGO_ENABLED=0 in the build stage above — it forces a static binary. If you genuinely need CGO, use the gcr.io/distroless/base variant (which carries glibc) rather than static, and match the glibc version between builder and runtime.
The empty-logs panic. When a distroless container crash-loops, docker logs may show nothing, and your instinct (“exec in and check”) is dead on arrival. Build the muscle memory now: a silent, instant-exit distroless container is almost always one of the three above — missing loader, missing certs, or missing a file the runtime expected. Check those before you reach for the debug container.
Interpreted runtimes are where it gets fiddly
The Go and Rust story is clean because a compiled static binary genuinely needs nothing else at runtime. Python and Node are a different proposition, and worth a frank warning. Google ships gcr.io/distroless/python3 and nodejs variants, and they work — but interpreted ecosystems lean on shelling out far more than people realise. A library that quietly calls subprocess.run(["git", ...]), an npm package whose post-install assumes /bin/sh, a healthcheck written as a shell one-liner: all of these fail in a distroless image, and the failure is usually a cryptic “file not found” rather than an honest “there is no shell.”
The Python variant also pins a specific interpreter version baked into the image, so you don’t choose your Python the way you would on a normal base — you take what the tag gives you, and you must match your build environment to it or your compiled C extensions won’t load. The practical advice: for interpreted services, prototype on a slim distro base (python:3.13-slim), get it working, then attempt the distroless swap and test the unhappy paths — error handlers, subprocess calls, healthchecks — explicitly. If the swap turns into a fight, a well-pruned -slim image with a non-root user and a scanned dependency tree is a perfectly respectable second place. Distroless is a tool, not a religion.
The size win, in proportion
Image size gets oversold as the headline benefit, so let’s keep it honest. Yes, a distroless Go image at a few megabytes versus a node:20 image at nearly a gigabyte is a dramatic ratio, and it does pay off: faster pulls on a cold node, less registry storage, quicker cold-starts in autoscaling. But on a homelab with a handful of long-lived containers, the size saving is mostly aesthetic — those images get pulled once and sit there for months. Don’t adopt distroless for the size; adopt it for the attack-surface reduction and treat the smaller image as a pleasant side effect. The security argument is the one that survives scrutiny; the size argument is the one that looks good in a blog post and rarely changes your day.
When the trade is worth it
I run distroless for anything that compiles to a static binary, because the cost is almost nil: Go and Rust don’t need a shell at runtime, the multi-stage build is barely more code, and the security and size wins are free money. For interpreted runtimes — Python, Node — distroless still works but the calculus is closer, because those ecosystems lean on shelling out more than people admit, and the occasional library that calls subprocess or expects /bin/sh will fail in confusing ways.
So, is it worth it? For production services, yes, with eyes open. A smaller image pulls faster, a smaller attack surface genuinely reduces risk, and “an attacker can’t get a shell” is a real mitigation, not security theatre. But adopt it as a team decision, not a unilateral one — the first colleague who tries to exec into a crashing prod pod at 3am and finds nothing there will not thank you for the surprise. Document the kubectl debug workflow, put it in the runbook, and practise it once before you need it. Distroless isn’t harder, exactly; it’s just differently hard, and the difficulty lands at the worst possible moment if you haven’t prepared. Prepared, it’s one of the cleanest security wins available.
For a homelab specifically, my honest recommendation is to start with the services that compile to a single static binary — anything in Go or Rust you already run — because the migration is nearly free and the payoff is real. Leave the interpreted services on a hardened -slim base until you have the spare evening to test their unhappy paths properly. That graduated approach gets you most of the benefit without betting an incident on an untested image, and it keeps the “differently hard” parts confined to the runs where you chose to take them on rather than discovering them at 3am.
Related Content
