Docker Compose Demystified: A Full Stack in a Single File

The fastest way to lose a weekend is to stand up a self-hosted app by hand, get it working at 2am, and then need to rebuild it six months later from memory. I’ve done it. The container’s exact flags — the volume path, the network name, the three environment variables it absolutely needed — lived only in my shell history and the increasingly unreliable part of my brain that thinks it remembers things. When the SD card died, so did all of it.

That is the problem Docker Compose exists to kill. If you’ve followed any of the self-hosting guides on this blog, you’ll have spotted the same quiet hero turning up again and again: a file called compose.yaml. That is no accident. Compose turns a sprawling mess of container commands into one readable blueprint you can start, stop, version, and reproduce byte-for-byte on a new machine. Understanding it properly makes every other containerised project click into place. This guide explains the problem Compose actually solves, walks the anatomy of a Compose file, builds a real three-service stack, covers the everyday commands, and — because nothing is ever as smooth as the tutorial — the things that go wrong and how to fix them.

The problem with juggling docker run

You can absolutely run containers by hand, and for a throwaway test you should. The trouble is what the command looks like once a container does anything useful. A single database with persistent storage, a custom network, environment variables, a restart policy and a published port becomes an unwieldy one-liner:

1
2
3
4

docker run -d --name db --restart unless-stopped \
  -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=app \
  -v db_data:/var/lib/postgresql/data \
  --network appnet -p 5432:5432 postgres:16

That’s one container. A real application has three or four, each with its own equally fiddly command, each needing to start in the right order with matching network names. Now reproduce that on another machine, six months later, from memory. The configuration lives nowhere reliable, which is to say it doesn’t really exist — you have a running system and no description of it. The day it breaks, you are reverse-engineering your own setup under pressure.

Compose fixes this by moving every one of those flags into a declarative file. Instead of remembering commands, you describe the desired state once and let Compose make it so.

What Compose is, and the mental shift

Docker Compose is a tool for defining and running multi-container applications. You write a YAML file describing your services, and Compose creates the networks, volumes and containers to match. It ships as a plugin to the Docker CLI, invoked as docker compose — two words. The old standalone docker-compose hyphenated binary (the Python v1 tool) is end-of-life; if a guide still tells you to install it, the guide is stale. Everything here uses the Compose V2 plugin, which is what a current Docker install gives you.

The real shift is from imperative to declarative. With docker run you issue commands that do things, in order, by hand. With Compose you describe what should exist, run one command, and Compose works out what to create, update or leave alone. The file becomes the single source of truth: commit it to version control and the system is now documented, diffable and reproducible. This is the same instinct behind infrastructure-as-code at larger scale — when you outgrow a single host and start eyeing an orchestrator, the leap to k3s on a Raspberry Pi is mostly a matter of translating these same concepts into a busier vocabulary.

Anatomy of a compose.yaml

A Compose file is a YAML document with a handful of top-level keys. The most important is services, under which each container is defined. The common building blocks per service:

• image — the container image to run, such as nginx:alpine. Pin a real tag, never trust latest for anything you care about.
• ports — published ports in host:container form, e.g. "8080:80".
• volumes — persistent storage or bind mounts, mapping a named volume or host path into the container.
• environment — environment variables passed into the container.
• networks — which networks the service joins, so containers find each other by name.
• depends_on — declares start-order, optionally gated on a healthcheck.
• restart — the restart policy, commonly unless-stopped.

Two more top-level keys, volumes and networks, declare the named volumes and networks your services reference. That’s genuinely most of it. The vocabulary is small, which is exactly why Compose is approachable in an afternoon where a full orchestrator is not.

A worked example: web app, database, reverse proxy

Theory is thin gruel, so let’s build something real: a web application, a PostgreSQL database behind it, and an Nginx reverse proxy in front. Create compose.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

services:
  proxy:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - web
    restart: unless-stopped

  web:
    image: ghcr.io/example/webapp:1.4.0
    environment:
      DATABASE_URL: "postgres://app:${DB_PASSWORD}@db:5432/app"
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_DB: app
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  db_data:

Note what’s happening. The web service reaches the database at the hostname db, because Compose puts all services on a shared default network where each is reachable by its service name — no IPs, no --link, no manual wiring. The proxy depends on the web app, and the web app waits for the database to report healthy before it starts. No remembered ordering, no remembered network names. The whole topology is right there in twenty lines you can read top to bottom.

Up, logs, exec and down

With the file written, the everyday commands are few. Build and start everything detached:

1

docker compose up -d

Compose creates the network, the named volume and all three containers in dependency order. See what’s running:

1

docker compose ps

Follow logs, optionally for one service:

1

docker compose logs -f web

Open a shell inside a running container — say, to poke at the database:

1

docker compose exec db psql -U app

And tear it all down:

1

docker compose down

That stops and removes the containers and the network but, crucially, leaves your named volumes intact. Add -v only if you genuinely want to delete the data too — a decision to make deliberately, never by reflex. I have nuked a database with a careless docker compose down -v exactly once, which is how I learned to treat that flag with suspicion.

Named volumes and persistence

Containers are ephemeral by design: delete one and anything written inside it vanishes. That’s fine for a stateless web server and catastrophic for a database. Volumes are how data survives the container’s lifecycle.

In the example, db_data is a named volume declared at the bottom of the file and mounted into Postgres’s data directory. Because Docker manages its lifecycle separately from the container, you can recreate, upgrade or restart the database container and the data persists:

1
2

docker volume ls
docker compose down && docker compose up -d   # data survives

The alternative is a bind mount, mapping a host directory like ./config:/etc/app, which is ideal for config files you want to edit on the host. The rule of thumb: named volumes for data the container owns, bind mounts for files you want to read and edit yourself. Mixing them up — bind-mounting a database directory, say — is a reliable source of permission headaches.

.env files, healthchecks and profiles

Three features turn a tidy Compose file into a properly maintainable one.

Environment files. Notice ${DB_PASSWORD} in the example. Compose automatically reads a file called .env in the same directory and substitutes those variables, keeping secrets and machine-specific values out of the committed YAML:

1
2

# .env
DB_PASSWORD=a-long-random-password

Add .env to .gitignore and your passwords stay out of version control while the structure stays shareable. Commit a .env.example with the keys but dummy values so the next person (often future-you) knows what to fill in.

Healthchecks. The database defines a healthcheck running pg_isready until the database genuinely accepts connections. This is what makes condition: service_healthy meaningful: the web app doesn’t merely wait for the database container to exist, it waits for the database to be ready, which eliminates a whole genre of start-up race conditions where the app crashes because Postgres wasn’t listening yet.

Profiles. Sometimes you want optional services — a debug tool, a one-off migration runner — that shouldn’t start by default. Tag them with a profile:

1
2
3

  backup:
    image: example/backup
    profiles: ["tools"]

A plain docker compose up -d ignores it; docker compose --profile tools up -d includes it. One file, several modes.

What goes wrong, and how to fix it

The happy path is short. The interesting part is the failures, and a handful account for most of them.

up hangs or a service restarts in a loop. Run docker compose ps — a container stuck in Restarting is crashing on boot. docker compose logs <svc> almost always tells you why: a missing env var, a config file that didn’t mount, a port already in use. Resist the urge to up -d repeatedly; read the log once.

“port is already allocated”. Something else on the host owns that port — often a previous Compose run that didn’t come down cleanly, or a system service. docker compose down first, then check with ss -tlnp what’s holding it. Change the host side of the mapping ("8081:80") if the conflict is genuine.

The app can’t reach the database. Nearly always a name or readiness problem. Confirm the app is using the service name (db), not localhost — localhost inside a container means that container, not the host or a sibling. And confirm the healthcheck-gated depends_on is in place, or the app may start before the database is listening.

Changes to the file don’t take. docker compose up -d reconciles, but it only recreates services whose definition changed. If you edited a bind-mounted config file rather than the Compose file, the container won’t restart on its own — docker compose restart <svc> forces it. If you genuinely want a clean rebuild of an image you build locally, docker compose up -d --build.

Permission denied on a volume. Usually a bind mount where the container’s user ID doesn’t match the host directory’s owner. Either align the IDs (many images take a PUID/PGID env var) or switch to a named volume and let Docker own the permissions. This is the single most common bind-mount trap.

When the answer to “why is it slow / leaking / unhappy” isn’t in the logs, the next move is metrics. Wiring a stack into Grafana and Prometheus is itself just another short Compose file, and it turns “I think it’s the database” into something you can actually see.

The everyday workflow, and the verdict

Put together, your daily rhythm with Compose is pleasingly dull, which is the highest praise infrastructure can earn. You edit compose.yaml, run docker compose up -d, and Compose reconciles reality with your description, recreating only what changed. You check docker compose logs -f when something misbehaves, drop into docker compose exec when you need a poke around, and docker compose pull followed by up -d to roll out new image versions.

Is it worth learning properly rather than copy-pasting commands? For anyone running more than one or two long-lived containers, unreservedly yes. Because the whole stack lives in one version-controlled file, rebuilding on a new server is a git clone and a single command away — the exact disaster I opened with becomes a non-event.

The honest limits: Compose orchestrates one host. It has no notion of scheduling across a cluster, rolling updates with no downtime, or self-healing across machines — for that you reach for an orchestrator, and even then you might find the added complexity isn’t worth it, the same way you might decide Podman without Docker or skipping Helm entirely is the saner call for a small setup. For a homelab, a small business, or any single beefy box doing real work, Compose is the sweet spot: powerful enough to define a real stack, simple enough to fit in your head. Master this one file and the rest stop being a pile of fiddly containers and become tidy, reproducible blueprints you can stand up anywhere in minutes.