Smallstep and mTLS Between Homelab Services

The uncomfortable truth about a typical homelab is that once something is on the network, it can talk to everything. Your Postgres container answers to anything that can reach port 5432. Your Prometheus scrapes endpoints that will happily dump metrics to whoever asks. The compromised smart plug on the same VLAN as your NAS has, from the network’s point of view, exactly the same standing as you do. We paper over this with firewall rules and the reasonable assumption that nothing hostile is inside — which is the assumption that stops being reasonable the day it stops being true.

Mutual TLS is the answer that scales down. Both ends of every connection present a certificate; both ends verify the other. Being on the network buys an attacker the ability to open a TCP socket and get hung up on. It is the same idea a service mesh sells you, minus the mesh, and with Smallstep’s step-ca doing the issuing it is achievable on a rack of compose files in an evening.

I want to be straight about this up front: mTLS is more work than it looks and less work than the service-mesh vendors imply. Here is the version that actually survives a homelab.

What mTLS changes, and what it does not

Ordinary TLS is one-directional identity. The server proves it is db.mylab.local by presenting a certificate signed by a CA the client trusts. The client proves nothing; it is anonymous until it types a password.

mTLS makes the client do the same thing. It presents its own certificate, signed by a CA the server trusts, and the server checks it before the connection completes. The result is that identity is established by the transport, before your application code runs, before a login form exists, before an unauthenticated code path can be reached. The vulnerable request never gets far enough to be a request.

Two things people expect from mTLS that they do not get:

It is authentication, and authorisation is your problem. A valid certificate proves who. It says nothing about what they may do. If every service in your lab has a certificate from the same CA, then every service can authenticate to every other service, and you have built a very sophisticated flat network. The authorisation step — checking the certificate’s Common Name or SAN against a list of who is allowed — is separate, and skipping it is the single most common way people get less than they think from this.

It does not stop a compromised service. If an attacker owns the box, they own the key on the box, and they can be that service. mTLS raises the cost of lateral movement from “scan the subnet” to “compromise a specific host and steal its key before it rotates”. That is a genuine improvement and it is worth the effort; treating it as a wall is how you end up disappointed. Set your expectations against an honest threat model rather than against marketing.

Certificates that renew themselves

The reason mTLS historically failed in small deployments is renewal. Hand out ten-year client certificates and you have created ten small time bombs and a spreadsheet. Hand out 24-hour certificates and you need automation, which is exactly what step-ca is for.

The mechanism is step ca renew --daemon. It sits beside a service, watches the certificate, and renews it at two-thirds of its lifetime — no cron, no timer, no drift:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Bootstrap a host's trust in the CA once.
step ca bootstrap --ca-url https://ca.mylab.local:8443 \
  --fingerprint 4a6f2c1d8e0b3f5a7c9d1e2f4a6b8c0d2e4f6a8b0c2d4e6f8a0b2c4d6e8f0a1b

# Get an identity for this service.
step ca certificate "metrics.mylab.local" \
  /etc/tls/svc.crt /etc/tls/svc.key \
  --provisioner homelab-services

# Then never think about it again.
step ca renew --daemon --exec "nginx -s reload" \
  /etc/tls/svc.crt /etc/tls/svc.key

The --exec hook is the part that matters. A renewed file on disk is useless if the daemon reading it parsed the old one at startup and has no idea. Some services watch the file, most do not, and finding out which is which is most of the work in a real deployment.

In compose, the pattern that has held up for me is a renewer sidecar sharing a volume:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

services:
  step-renewer:
    image: smallstep/step-ca:latest
    restart: unless-stopped
    entrypoint: /bin/sh
    command: >
      -c "step ca renew --daemon --renew-period 8h
          /certs/svc.crt /certs/svc.key"
    volumes:
      - certs:/certs
      - ./step-config:/home/step/config:ro
    environment:
      - STEPPATH=/home/step

  app:
    image: example/app:1.4.2
    restart: unless-stopped
    depends_on: [step-renewer]
    volumes:
      - certs:/certs:ro
    environment:
      - TLS_CERT=/certs/svc.crt
      - TLS_KEY=/certs/svc.key
      - TLS_CLIENT_CA=/certs/root_ca.crt

volumes:
  certs:

This works, and it has a flaw I will not pretend away: the renewer needs a provisioner credential, and that credential lives in the compose stack. Anything that can read the volume can ask for a certificate. In a homelab I accept this; the alternative is a proper workload-identity system and that is a genuinely large project.

Enforcing it at the proxy

Configuring mTLS in a dozen applications individually is miserable, because every application has a different idea of what a client CA setting is called and about a third of them do not have one. Terminate at the proxy instead, and make the proxy do the authorisation you were going to skip.

Caddy is the least painful:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

metrics.mylab.local {
	tls /certs/svc.crt /certs/svc.key {
		client_auth {
			mode require_and_verify
			trust_pool file /certs/root_ca.crt
		}
	}

	# Authentication is done. Now authorisation.
	@allowed {
		expression {tls_client_subject_cn}.endsWith("prometheus.mylab.local")
	}
	handle @allowed {
		reverse_proxy 127.0.0.1:9090
	}

	handle {
		respond "certificate valid, access denied" 403
	}
}

require_and_verify is the whole point — request mode asks for a certificate and shrugs when it does not get one, which is a configuration that looks like mTLS in a diagram and is not mTLS in reality. Check for that mode specifically when reviewing anyone’s setup, including your own from six months ago.

The @allowed matcher is the authorisation step. Without it, any certificate from your CA opens the door, including the one on the Jellyfin box that has no business reading your metrics. With it, you have per-service policy expressed in the same file as the routing, and that is about as good as it gets without adopting something much bigger.

Nginx does the same thing more verbosely:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

server {
    listen 8443 ssl;
    server_name metrics.mylab.local;

    ssl_certificate     /certs/svc.crt;
    ssl_certificate_key /certs/svc.key;
    ssl_client_certificate /certs/root_ca.crt;
    ssl_verify_client on;
    ssl_verify_depth 2;

    # Authorisation, since a valid cert alone is not authority.
    if ($ssl_client_s_dn !~ "CN=prometheus\.mylab\.local") {
        return 403;
    }

    location / { proxy_pass http://127.0.0.1:9090; }
}

ssl_verify_depth 2 matters with a two-tier CA. The default of 1 will reject every certificate you issue, and the error message points at the certificate rather than at the depth setting, which is how you lose an hour.

Where to start, and where to stop

Do not attempt to mTLS everything. That project has a failure rate near 100% and its usual outcome is a network where half the things are broken and the other half have verify off set as a temporary measure two years ago.

Pick the connections where the value is highest and the endpoints are few:

• Database connections. Postgres and MariaDB both support verify-full client certificates natively, the client list is short, and the prize is that a stolen application password is now insufficient on its own.
• Metrics scraping. Prometheus supports client certificates per scrape config, exporters leak more than people realise, and this is a small, contained, high-value change.
• The backup path. Whatever pushes your data off-site is a connection you would very much like to be sure about at both ends.
• Anything crossing a segment boundary. If you already have VLANs, the traffic that crosses between them is exactly the traffic worth authenticating.

Leave alone: anything with a single user, anything already behind a login you trust, and anything whose client is a browser. Browser mTLS means installing a client certificate on every device, and the user experience is a native certificate-picker dialog that appears at the wrong moment and cannot be styled or explained. It is a fine choice for a two-device admin interface and a bad one for anything a family member touches.

A worked example: locking down Postgres

Abstract advice is easy to nod at, so here is the one I would actually do first, end to end. Postgres is a good starting point because it speaks mTLS natively, the client list is short and enumerable, and the improvement is concrete.

Issue the server an identity and give the database a client CA to check against:

1
2
3
4
5
6

# postgresql.conf
ssl = on
ssl_cert_file = '/certs/svc.crt'
ssl_key_file  = '/certs/svc.key'
ssl_ca_file   = '/certs/root_ca.crt'
ssl_min_protocol_version = 'TLSv1.3'

Then the part that does the real work, in pg_hba.conf:

1
2
3
4

# TYPE  DATABASE   USER      ADDRESS         METHOD
hostssl  grafana    grafana   192.168.1.0/24  cert clientcert=verify-full
hostssl  paperless  paperless 192.168.1.0/24  cert clientcert=verify-full
host     all        all       0.0.0.0/0       reject

cert clientcert=verify-full is doing two jobs at once. It requires a valid client certificate from your CA, and it requires the certificate’s Common Name to equal the Postgres username being requested. The grafana role can now only be used by a client holding a certificate issued for grafana. There is no password at all in this configuration — the certificate is the credential, it expires by itself, and the .env file full of database passwords that half your stack shares can be deleted.

The client side, for a service that does not know about step:

1
2

step ca certificate "grafana" /certs/grafana.crt /certs/grafana.key   --provisioner homelab-services --not-after 24h
chmod 0600 /certs/grafana.key   # libpq refuses a group-readable key, silently-ish

1
2

# Connection string. sslmode=verify-full is the only value that means anything.
postgresql://[email protected]:5432/grafana?sslmode=verify-full&sslrootcert=/certs/root_ca.crt&sslcert=/certs/grafana.crt&sslkey=/certs/grafana.key

The trap is sslmode. require encrypts and verifies nothing — it will happily hand your credentials to any server that answers, which makes it security theatre with a performance cost. verify-ca checks the signature and ignores the hostname. Only verify-full does what everyone assumes require does. This has been true and misunderstood for twenty years.

Roll it out one service at a time by leaving the old md5 line in pg_hba.conf above the new cert line while you migrate, then delete it. Reload with SELECT pg_reload_conf(); and watch pg_stat_ssl to confirm what is actually happening rather than what you believe is happening:

1
2

SELECT c.usename, s.ssl, s.version, s.client_dn
FROM pg_stat_ssl s JOIN pg_stat_activity c USING (pid);

An empty client_dn on a connection you thought was migrated means that service is still using the password path and you have not finished.

The case against doing any of this

I would rather make the argument myself than have you make it at me.

mTLS adds a hard dependency on the CA to every connection in the lab. It adds a renewal daemon per service, a volume per service, and a class of outage — “the clock is wrong” — that most homelabs have never had to think about. Debugging a handshake failure is meaningfully harder than debugging a wrong password, and the error messages are worse on purpose. When it breaks it tends to break for everything simultaneously, at whatever hour the certificates happened to be issued.

Against that, the realistic threat to a home network is a vulnerable service exposed to the internet, or a supply-chain compromise in a container image you pulled without looking. mTLS between internal services does very little about either. It addresses lateral movement, which is step two of an attack whose step one you should be preventing with updates and by not exposing things.

So: if your services are still running as root, if your backups are untested, if you are running :latest and pulling nightly — do those first. They are cheaper and they buy more. mTLS is a good third or fourth move and a poor first one.

Troubleshooting

tls: bad certificate and nothing else. TLS alerts are deliberately uninformative to the other end. Read the server’s logs, and if those are silent, watch the handshake directly:

1
2
3
4

step certificate inspect --insecure /certs/svc.crt
openssl s_client -connect metrics.mylab.local:8443 \
  -cert /certs/client.crt -key /certs/client.key \
  -CAfile /certs/root_ca.crt -showcerts

The Verify return code at the bottom of that output is the answer. 21 (unable to verify the first certificate) means a chain problem; 18 (self signed certificate) means the client does not trust your root; 0 (ok) with a rejection afterwards means your authorisation rule is the thing saying no.

Works from the host, fails from a container. Container clocks drift, and a certificate valid for 24 hours is unforgiving about a machine that thinks it is Tuesday. Check date inside the container. This accounts for a startling proportion of mTLS mysteries.

Everything breaks at once, at 4am, after months. The intermediate certificate expired, or the CA was down when the renewal window opened and the certificates simply ran out. Twenty-four-hour lifetimes mean a CA outage becomes a total outage within a day. Monitor the CA more aggressively than anything it serves, and alert on renewal age rather than on certificate expiry — by the time expiry alerts, you have hours.

Prometheus reports x509: certificate is valid for X, not Y. The SAN does not match the name you connected to. Issue certificates with every name and IP that will be used to reach the service, including the container name if that is how the scrape config addresses it.

A service ignores the renewed certificate. It cached the old one at startup. --exec a reload, verify with openssl s_client that the served notAfter actually moved, and if reload does not do it, restart. Assume nothing here.

The verdict

mTLS with step-ca is the right call for a specific, defensible set of connections in a homelab: databases, metrics, backups, anything crossing a trust boundary. It costs an evening for the first service and about ten minutes for each one after that, and it converts “on the network” from an authorisation into a mere fact of topology. The automatic renewal is what makes it viable at this scale, and it genuinely works — my certificates rotate several times a day and I have not thought about them in months.

It is the wrong call as a blanket policy, wrong for browser-facing services, and wrong if you skip the authorisation step, which turns a security control into an elaborate way of proving that your own machines are your own machines. If you have already done network segmentation and it is holding, mTLS is the layer that makes segmentation survive being wrong. Start with the database. See how it feels in a month.