Ollama Behind a Reverse Proxy With Real Authentication

Ollama has no authentication. None. There is no password, no token, no user list, no config file where you enable one. This surprises people, and then they discover the fix that seems obvious — set OLLAMA_HOST=0.0.0.0 so the laptop can reach it — and now every device on the network can pull models, delete models, and burn your GPU on someone else’s prompts.

This is not a bug. Ollama’s threat model is “a program running on your machine, listening on loopback”, the same model as your local Postgres or your dev server. Loopback-only is a perfectly coherent security boundary. It stops being coherent the moment you change the bind address, and changing the bind address is the first thing everybody does.

So: if you want to reach your models from another machine, something in front of Ollama has to do the authenticating. Here’s what I run, and why the obvious shortcuts didn’t survive contact with reality.

What an open port 11434 actually gives away

Worth being concrete about the stakes, because “an unauthenticated API” sounds abstract until you read the endpoint list:

• POST /api/generate and /api/chat — run inference. Somebody else’s prompts, your electricity, your GPU. Also your logs, if you keep any.
• POST /api/pull — download an arbitrary model from the registry onto your disk. A 40 GB model is one curl away. Repeat until the filesystem is full and everything else on the box falls over.
• DELETE /api/delete — remove any model you’ve downloaded.
• POST /api/create — create a model from a Modelfile, which includes a FROM pointing at a local path.
• GET /api/tags — enumerate everything you have.

The interesting one is /api/pull. Every other endpoint costs you compute; that one costs you disk, silently, at whatever speed your line runs. It’s the endpoint I’d reach for if I found an open Ollama on a network and wanted to be annoying rather than clever.

Two shortcuts, and why I stopped using them

Shortcut one: bind to 0.0.0.0 and firewall the port. Source-IP restriction is real security, and if your only client is one fixed workstation this genuinely works. It stopped working for me the day I wanted to hit the API from a phone, then from a VM whose address changes, then from a container. Every one of those is a new firewall rule, and the rules accumulate faster than the discipline to prune them.

Shortcut two: put it on the mesh VPN and call it done. Tailscale is genuinely excellent and I use it for exactly this kind of problem. The gap: on the tailnet, every device I own is authenticated, so my compromised smart TV and my laptop have identical rights to the inference API. Mesh VPN gets you network authentication. Application authentication is a separate question and the VPN has an opinion about neither users nor rate limits.

Both shortcuts are fine for some threat models. Mine involves a house with a lot of devices in it, so I wanted the API to know who was asking.

The shape of the answer

Two lanes, because there are two kinds of client and they authenticate differently:

• Browsers get redirected to a login page, get a session cookie, come back. This is forward auth, and Authelia does it well.
• Scripts cannot follow a login redirect. They get a static bearer token checked at the proxy.

Caddy handles both with a header matcher, and it does its own TLS, which removes an entire category of certificate chores.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

# docker-compose.yml
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ollama-models:/root/.ollama
    networks: [ai]
    # no ports: — nothing outside this network reaches 11434
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  caddy:
    image: caddy:2.7
    container_name: caddy
    ports:
      - "443:443"
      - "80:80"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy-data:/data
    networks: [ai]
    depends_on: [ollama, authelia]

  authelia:
    image: authelia/authelia:4.37
    container_name: authelia
    volumes:
      - ./authelia:/config
    networks: [ai]

volumes:
  ollama-models:
  caddy-data:

networks:
  ai:

The line doing the most work is the one that isn’t there: ollama has no ports: mapping. It’s reachable only from inside the ai Docker network, which means the reverse proxy is the sole path in. If you leave ports: - "11434:11434" on the container while adding a beautiful auth layer beside it, you have built a front door next to an open window. I have done this. Twice.

The Caddyfile

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

{
	email [email protected]
}

# The login portal itself
auth.mylab.local {
	reverse_proxy authelia:9091
}

ai.mylab.local {
	# Lane 1: scripts presenting a bearer token skip the portal entirely
	@apitoken header Authorization "Bearer {env.OLLAMA_API_TOKEN}"
	handle @apitoken {
		reverse_proxy ollama:11434 {
			flush_interval -1
			transport http {
				read_timeout 600s
			}
		}
	}

	# Lane 2: everything else goes through Authelia
	handle {
		forward_auth authelia:9091 {
			uri /api/verify?rd=https://auth.mylab.local
			copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
		}
		reverse_proxy ollama:11434 {
			flush_interval -1
			transport http {
				read_timeout 600s
			}
		}
	}
}

Three details that took me longer than they should have:

flush_interval -1 disables response buffering. Ollama streams tokens as newline-delimited JSON. Without this, Caddy helpfully buffers the response and delivers the entire generation in one lump at the end, which looks exactly like the model hanging for forty seconds and then finishing instantly. Every “streaming doesn’t work behind my proxy” thread ends here.

read_timeout 600s because the first request against a cold model includes loading several gigabytes from disk into VRAM. The default timeout will cut that off and hand you a 502 while the load is still perfectly healthy underneath.

The @apitoken matcher does an exact string comparison on the whole header value. Caddy’s header matcher is not constant-time, so this is technically timing-attackable; against a random 32-byte token over a network that leaks microseconds, I’m comfortable. If you aren’t, put the token check in Authelia and use its API instead.

Authelia’s half of the deal

Caddy asks “is this request authenticated?” and Authelia answers. The answer comes from an access-control policy, and the default posture matters more than the rules you write after it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

# authelia/configuration.yml (abridged)
theme: dark
default_redirection_url: https://auth.mylab.local

server:
  host: 0.0.0.0
  port: 9091

authentication_backend:
  file:
    path: /config/users.yml
    password:
      algorithm: argon2id
      iterations: 3
      memory: 65536
      parallelism: 4

access_control:
  default_policy: deny
  rules:
    - domain: auth.mylab.local
      policy: bypass
    - domain: ai.mylab.local
      policy: two_factor
      subject:
        - "group:llm-users"

session:
  name: authelia_session
  domain: mylab.local
  expiration: 12h
  inactivity: 45m

storage:
  local:
    path: /config/db.sqlite3

notifier:
  filesystem:
    filename: /config/notification.txt

default_policy: deny is the line I’d fight for. Start closed and open specific holes, because a policy that defaults to one_factor will happily protect a service you forgot you were exposing with a password you set in 2019.

two_factor on the AI host is a judgement call. My inference endpoint holds nothing secret in itself, and the session lasts twelve hours, so the TOTP prompt shows up roughly once a day. That’s a price I’ll pay to keep the habit of second factors on everything that faces the network. The notifier: filesystem block is fine for a home setup where you’re the only user — Authelia writes the enrolment link to a text file you cat once during setup.

Passwords go in users.yml, hashed with Authelia’s own tool so you never paste a plaintext into a config:

1
2

docker run --rm -it authelia/authelia:4.37 \
  authelia hash-password -- 'a long passphrase you will actually remember'

Generating the script token, and rate-limiting the lane

The bearer token wants to be long, random, and stored somewhere your scripts can read without you copying it around:

1
2
3

# 32 bytes, URL-safe, no shell-hostile characters
openssl rand -base64 32 | tr -d '=+/' | cut -c1-40 > /srv/ai/.ollama_token
chmod 600 /srv/ai/.ollama_token

Caddy reads it from the environment, so it lands in the compose file’s environment: block for the caddy service and never inside the Caddyfile itself. If you keep the compose stack in git — and you should — that token belongs in SOPS with an age key rather than in the repo as plain text.

Rate limiting is the part I skipped for six months and regret skipping. A misbehaving script can queue up requests faster than the GPU retires them, and Ollama’s queue is not visible from outside; from the client’s perspective, everything just gets slower until it times out. Caddy’s rate-limit module isn’t in the standard build, so it needs xcaddy:

1
2
3
4
5

FROM caddy:2.7-builder AS builder
RUN xcaddy build --with github.com/mholt/caddy-ratelimit

FROM caddy:2.7
COPY --from=builder /usr/bin/caddy /usr/bin/caddy

1
2
3
4
5
6
7

	rate_limit {
		zone inference {
			key    {http.request.header.Authorization}
			events 20
			window 1m
		}
	}

Twenty generations per minute per token is generous for a human and cripplingly restrictive for a runaway loop, which is exactly the discrimination I want. Keying on the Authorization header rather than the client IP means one misbehaving script gets throttled while other clients on the same NAT keep working.

Locking down the pull endpoint

Even authenticated, I don’t want a stray script filling my disk. Caddy can refuse the destructive endpoints outright for the token lane:

1
2
3
4
5
6

	@dangerous {
		path /api/pull /api/delete /api/create /api/push
	}
	handle @dangerous {
		respond "model management is not available over this endpoint" 403
	}

Put that block before the lanes. Managing models is then something I do by exec-ing into the container, which is the right amount of friction for an operation that writes tens of gigabytes:

1
2

docker exec -it ollama ollama pull mistral:7b-instruct-q4_K_M
docker exec -it ollama ollama list

The option I keep not taking: mTLS

There’s a purer answer to all of this. Give every client a certificate, make Caddy demand one, and no unauthenticated request ever reaches the application layer at all:

1
2
3
4
5
6
7
8
9

ai.mylab.local {
	tls internal {
		client_auth {
			mode require_and_verify
			trusted_ca_cert_file /etc/caddy/client-ca.pem
		}
	}
	reverse_proxy ollama:11434
}

Four lines, no login portal, no session store, no token to leak in a shell history. Cryptographically it beats everything above.

I run it for machine-to-machine paths and I have given up trying to run it for anything a human touches. Loading a client certificate into a phone browser is a small ordeal; doing it on a work laptop you don’t fully control is worse; renewing them all in a year’s time is a chore you will have forgotten how to do. The honest accounting is that mTLS is the better protocol and the worse product, and if you want to try anyway, run your own CA properly rather than hand-rolling openssl commands you’ll never repeat correctly.

Troubleshooting

Everything returns 502 immediately. Caddy can’t resolve ollama:11434. The containers are on different networks, or you named the service something else. docker exec caddy nslookup ollama settles it in one command.

Streaming arrives all at once. flush_interval -1, as above. If you’re on nginx instead, the equivalent is proxy_buffering off; plus proxy_cache off;.

First request 502s, second one works. Cold model load exceeded the read timeout. Raise it, or pre-warm at startup with a one-token generate against the model you actually use.

Authelia redirects in a loop. Almost always the cookie domain in configuration.yml doesn’t cover the subdomain you’re protecting, so the session cookie is set and then invisible on the next request. session.domain has to be the parent domain (mylab.local), and both hosts have to sit under it.

A browser-based UI works but its API calls fail. Front-end clients send OPTIONS preflights that carry no Authorization header, so they land in the Authelia lane and get a redirect, which the browser reads as a CORS failure. Handle OPTIONS explicitly before the lanes:

1
2
3
4
5
6

	@preflight method OPTIONS
	handle @preflight {
		header Access-Control-Allow-Origin "https://chat.mylab.local"
		header Access-Control-Allow-Headers "Authorization, Content-Type"
		respond 204
	}

Tokens work from curl but fail from Python. Some HTTP clients normalise header capitalisation; Caddy’s matcher is case-insensitive on the header name and case-sensitive on the value. Check you aren’t sending bearer lowercase.

Generations get slower for everyone when one client is busy. That’s the queue, and it’s the symptom the rate limit exists to convert into an honest 429. Ollama serves requests against a loaded model sequentially, so a client asking for six long completions has effectively taken the GPU for the next few minutes. Watching docker logs -f ollama during the slowdown shows the requests arriving and sitting.

Is it worth it?

If Ollama runs on the same laptop you type on, no. Loopback is already the boundary and adding a proxy adds failure modes for zero benefit. Leave it alone.

If Ollama runs on a box in a cupboard and anything else talks to it over the network, then yes — and the work is smaller than it looks. The compose file and the Caddyfile above took an evening, most of it spent on the streaming-buffer thing. What I got was a single TLS endpoint, one place where sessions are decided, and a model server that can’t be casually enumerated by whatever else is on the network.

The thing I’d emphasise to anyone starting: delete the ports: mapping from the Ollama container first, before you build anything else. Once nothing can reach 11434 directly, every subsequent mistake you make is a mistake in the proxy config, and those fail closed. It’s the same reasoning as putting everything else behind one reverse proxy — the value comes from there being exactly one way in.