MinIO and the S3-Compatible Homelab

Somewhere around 2015, S3 stopped being a product and became a protocol. That is the fact that makes an object store worth running at home. Look at what expects an S3 endpoint now: Restic and Kopia for backups, Litestream for SQLite replication, Loki for log chunks, Thanos for Prometheus history, Terraform for state, Docker registries, Gitea’s LFS, Nextcloud’s primary storage, half of every self-hosted app’s file uploads. All of them will happily talk to a bucket on a machine in your hallway, because none of them can tell the difference.

That is the pitch. An S3 endpoint on your own hardware is a universal adapter that a decade of software has already been written against. MinIO is the best-known way to provide one, and it is also the one with the biggest asterisk attached, so let us deal with the asterisk first.

The licence question, honestly

MinIO is AGPLv3. This matters far more than people initially assume, and pretending otherwise does nobody any favours.

For a homelab, AGPL is a non-issue. You run it, you use it, nobody is a “user over a network” in the sense the licence cares about, and there is no obligation you can plausibly trip over. Run it and stop worrying.

For anything commercial, read the licence properly or talk to someone who has. The AGPL’s network clause means that if you offer a service to third parties over a network and MinIO is part of it, the copyleft can reach into your stack in ways that surprise people who assumed “we didn’t modify it, so we’re fine”. MinIO also removed a substantial amount of functionality from the community build of its browser console, pushing the management UI toward the commercial offering — a strategy that is entirely their right and that has, predictably, sent a chunk of the community looking at alternatives. A visible chunk of that traffic has gone toward Garage and SeaweedFS, both of which are lighter, less compatible, and licensed more permissively.

I keep MinIO because it is fast, the CLI is excellent, and the compatibility surface is the widest of anything self-hostable. Those are real advantages and I want them stated plainly alongside the licensing caveat.

What object storage is for, and what it isn’t

An object store is a flat namespace of immutable blobs addressed by key, with metadata, versioning and an HTTP API. There is no filesystem underneath it that you should ever touch, and no partial writes — you replace an object wholesale.

That immutability is the actual feature. A backup target where the client cannot modify existing objects is a backup target that survives your ransomware, your rm -rf and your 2 a.m. cleanup script. Combine object locking with versioning and you have a genuinely append-only store, which is the property that makes the difference between a backup and a copy. The distinction is the entire argument of snapshots are not backups, arriving from a different direction.

What it is bad at: anything expecting POSIX semantics. Do not mount a bucket as a filesystem and run a database on it. Do not point your media library at it and expect seeks to be free. Every S3-as-a-filesystem layer works well enough in demos and badly under load, because you are emulating random writes on top of a store that fundamentally does not have them. Use S3 for the workloads written for S3, and there are now enough of those to justify the box on its own.

Running it

The single-node, single-drive deployment is genuinely two minutes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

# compose.yaml
services:
  minio:
    image: quay.io/minio/minio:latest
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: admin
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:?set me in .env}
      # Behind a reverse proxy on your own domain:
      MINIO_SERVER_URL: https://s3.example.com
      MINIO_BROWSER_REDIRECT_URL: https://s3-console.example.com
    volumes:
      - /srv/minio/data:/data
    ports:
      - "127.0.0.1:9000:9000"
      - "127.0.0.1:9001:9001"
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 30s
      timeout: 10s
      retries: 3

Binding to 127.0.0.1 and terminating TLS at a reverse proxy is the right shape here; the alternative is managing certificates inside the container for no benefit. MINIO_SERVER_URL matters more than it looks — MinIO signs presigned URLs against it, and getting it wrong produces presigned links that 403 in ways that take an hour to diagnose.

Then the client, which is where MinIO genuinely shines:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

mc alias set home https://s3.example.com admin "$MINIO_ROOT_PASSWORD"

# A bucket that a backup client can write to but never rewrite
mc mb --with-lock home/backups
mc version enable home/backups
mc retention set --default GOVERNANCE 30d home/backups

# A scoped user, because your backup client should not hold root
mc admin user add home restic-svc "$(openssl rand -base64 24)"
mc admin policy create home restic-write /tmp/restic-policy.json
mc admin policy attach home restic-write --user restic-svc

The policy is standard IAM JSON, which is another benefit of the protocol having won — the syntax is the same one you already half-know:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject", "s3:ListBucket",
                 "s3:GetBucketLocation", "s3:ListBucketMultipartUploads",
                 "s3:ListMultipartUploadParts", "s3:AbortMultipartUpload"],
      "Resource": ["arn:aws:s3:::backups", "arn:aws:s3:::backups/*"]
    }
  ]
}

Note what is missing: s3:DeleteObject. A backup client that cannot delete is a backup client that cannot be turned against you, and with retention set to GOVERNANCE 30d, an attacker holding those credentials can add objects and can do nothing about the ones already there. Pointing Restic and rclone at a bucket configured this way costs ten minutes and upgrades the whole arrangement.

Restic then needs three environment variables and nothing else:

1
2
3
4

export RESTIC_REPOSITORY="s3:https://s3.example.com/backups/nodes"
export AWS_ACCESS_KEY_ID="restic-svc"
export AWS_SECRET_ACCESS_KEY="..."
restic init && restic backup /srv --verbose

The workloads that justify the box

The reason this earns a permanent place is that it collapses several unrelated problems into one. Some that have paid for mine:

SQLite replication. Litestream streams the WAL of a SQLite database to a bucket continuously, giving you point-in-time recovery on a database that has no replication of its own. It wants an S3 endpoint and nothing else, and the recovery window drops from “last night’s backup” to “about a second ago”.

Prometheus history. Local Prometheus storage is designed for weeks rather than years. Point Thanos or a remote-write receiver at a bucket and your metrics retention becomes a question of disk price. That backup and retention thinking is the same discipline as the 3-2-1 backup rule, applied to observability data.

Loki chunks. Log aggregation on a filesystem works and gets awkward past a few hundred gigabytes. On object storage it stops being a capacity problem.

Terraform state. Versioned, locked, and no longer sitting in a directory you might accidentally delete.

None of these individually justifies a machine. Together they mean the object store is load-bearing within a month, and that is the pattern worth noticing: the value comes from the interface being shared, so each new consumer costs a bucket and a policy rather than a design.

Erasure coding, and whether you want it

MinIO’s headline feature is erasure coding across drives: split each object into data and parity shards so the store survives drive loss without RAID underneath. On a single-node multi-drive setup with four disks in the default EC:2 layout, you can lose two and keep serving.

Here is the part the documentation is quiet about. Erasure coding requires a minimum drive count, the layout is decided at deployment and cannot be changed afterwards, and expanding means adding an entire new server pool rather than a disk. That last constraint is brutal in a homelab, where growth is exactly one drive at a time as the budget allows. A four-drive erasure set that fills up wants four more drives, today, in one purchase.

So my honest recommendation for most homelabs: run MinIO single-drive on top of a filesystem that already handles redundancy. Let ZFS or your existing array do the parity, checksumming and scrubbing — jobs it does better and has done for longer — and let MinIO be a protocol layer. You lose MinIO’s distributed story, which you were not going to use, and you keep the ability to add one disk. The layout logic behind that choice is the same one in rethinking RAID.

The parts people forget

Set a lifecycle policy on day two. Multipart uploads that fail leave orphaned parts behind, invisible to mc ls, consuming space forever. A rule that aborts incomplete uploads after seven days stops a slow leak that otherwise takes a year to notice:

1
2

mc ilm rule add --expire-delete-marker --noncurrent-expire-days 90 home/backups
mc admin config set home api abort_incomplete_multipart_upload=24h

Back up the object store itself. This sounds circular and it isn’t. The bucket holding your backups is a single point of failure until it is replicated somewhere, and mc mirror --watch to a second machine or rclone sync to a cloud provider closes that gap. An object store is a destination rather than a strategy.

Give it its own credentials per consumer. Root credentials that end up in six .env files are six chances to leak them and no way to know which one did. A user per client, a policy per user, takes two minutes each and makes rotation possible.

Troubleshooting

Presigned URLs return 403 or point at 127.0.0.1. MINIO_SERVER_URL is unset or wrong, or your proxy is rewriting the Host header. MinIO computes signatures over the host it believes it is, and a mismatch fails the signature check. Make sure the proxy passes the original Host through unmodified.

SignatureDoesNotMatch on a client that worked yesterday. Clock skew, nine times out of ten. S3 signatures embed a timestamp with a 15-minute tolerance window, and a container whose host has drifted will fail every request with an error that says nothing about time. Check NTP before you check anything else.

Uploads over ~5 GB fail or stall. Multipart. The client is chunking, and either the proxy has a body size limit or the ListMultipartUploadParts/AbortMultipartUpload actions are missing from your policy. Both produce failures that surface at the end of a long upload, which is a maximally annoying place for them to surface.

The console shows almost nothing useful. That is deliberate — features moved to the commercial edition. mc admin covers essentially everything the console used to, and if you find yourself missing the UI badly, that is a genuine signal to evaluate alternatives rather than a bug.

Performance is dreadful on many small objects. Object stores have per-object overhead, and a workload of ten million 4 KB objects is the pathological case; every one is a separate HTTP request with signature verification attached. This is a real architectural limit rather than a tuning problem, and an entire category of stores exists to address it by packing small objects into large volumes. If your workload looks like this, MinIO is the wrong tool and no configuration will rescue it.

The verdict

An S3 endpoint is one of the highest-leverage things in a homelab, because it is the interface a decade of software already speaks. Backups, log storage, database replication, registries and CI artefacts all stop being individual problems and become one problem you have solved once. Mine has been running for years and is the box I would rebuild first.

MinIO specifically is the right pick when you want maximum compatibility, an excellent CLI, and you are running it for yourself under a licence whose implications end at your front door. Skip it if you have commercial ambitions you haven’t checked with a lawyer, if you need the browser console to do real work, or if your hardware grows one disk at a time and you were seduced by the erasure coding — in which case run it single-drive over ZFS, or run Garage.

The one thing I would tell my earlier self: set up the locked, versioned, delete-denied bucket on day one. It is fifteen minutes of mc commands, and it is the difference between an object store and an actual safety net.