Immich Machine Learning on a Spare GPU

Immich’s smart search is the feature that makes it worth leaving Google Photos for. You type “red bicycle in snow” and it finds the picture, because every photo in the library has been through a CLIP model and turned into a vector that sits near the vector for that sentence. Same for faces: an embedding per detected face, clustered, and now your library knows who’s in it.

The catch arrives when you point it at a real library. Eighty thousand photos, on the mini PC that runs everything else in my flat, meant an estimated completion time I initially assumed was a bug. Nine days. The machine would also be doing nothing else useful for those nine days, since ML jobs will happily eat every core you own.

The fix is structural and it’s the best decision in Immich’s architecture: the machine learning is a separate container speaking HTTP. It doesn’t have to live on the same machine as the server. Point it at a box with a graphics card in it and the nine days become an afternoon.

How the pieces fit

An Immich deployment is four containers, and it’s worth being clear about which does what before you start moving them around.

• immich-server — the API, the web app, thumbnail generation, video transcoding. Wants CPU and fast storage.
• immich-machine-learning — CLIP embeddings and face detection/recognition. Wants a GPU, or a great deal of patience.
• postgres — with a vector extension, because similarity search over embeddings is the whole feature. Immich ships pgvecto.rs for this.
• redis — the job queue.

The server talks to the ML container over plain HTTP, and where that container lives is entirely up to you. IMMICH_MACHINE_LEARNING_URL is one environment variable, and it accepts any URL. That’s the seam.

I have a machine with a graphics card in it that spends most of its life idle. My photos live on a different machine, which has no card and no free slot for one. Two boxes, one environment variable, and both get to be good at what they’re good at — the same shape of argument that had me passing a GPU through to a VM rather than reorganising a whole host around one card.

The ML container, on the GPU box

Immich publishes CUDA, ROCm and OpenVINO variants of the ML image. On the GPU machine, that’s a standalone compose file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# /srv/immich-ml/docker-compose.yml — on the machine with the card
services:
  immich-machine-learning:
    container_name: immich-ml
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    volumes:
      - model-cache:/cache
    environment:
      - MACHINE_LEARNING_WORKERS=1
      - MACHINE_LEARNING_MODEL_TTL=600
    ports:
      - "192.168.1.20:3003:3003"
    restart: always
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  model-cache:

Then on the server box, in its .env:

1

IMMICH_MACHINE_LEARNING_URL=http://192.168.1.20:3003

Restart the server, and that’s the migration. It genuinely is that small.

Three things about that compose file deserve a comment.

The -cuda tag. The default image is CPU-only and will silently work if you use it, at CPU speed, which is a confusing failure because nothing errors. docker logs immich-ml on startup tells you which execution providers ONNX Runtime found — look for CUDAExecutionProvider in the list rather than assuming.

MACHINE_LEARNING_WORKERS=1. More workers means more concurrent requests and more copies of the model in VRAM. On a card with room for one, a second worker turns a fast job into an out-of-memory crash halfway through. Raise it only after you’ve watched nvidia-smi during a full run and seen the headroom.

MACHINE_LEARNING_MODEL_TTL=600. Models unload from memory after ten minutes idle. Since this card does other work, I want Immich to let go of it when it’s finished. Set it to 0 to keep models resident permanently — right if the card is Immich’s alone, wrong if anything else wants it.

The bind on 192.168.1.20:3003 rather than 0.0.0.0 is deliberate. The ML API has no authentication of any kind and will accept an image from anyone who can reach it. Keep it on a network segment you trust, and if the two machines are further apart than one switch, put the traffic on a mesh VPN rather than exposing that port.

Choosing a model, which matters more than the GPU

Immich’s default CLIP model is ViT-B-32__openai. It’s small, fast, and the accuracy ceiling on your search quality. Changed in Administration → Settings → Machine Learning → Smart Search:

Once you have a GPU doing the work, the calculus flips completely. On CPU, the small model is the only tolerable option and you pick it under duress. On a card, the difference between ViT-B-32 and ViT-L-16-SigLIP-384 is a few hours of one-off indexing, and you keep the improvement forever. I run the SigLIP-384 and searches that used to return “something vaguely orange” now return the actual thing.

The catch worth knowing before you start: changing the model invalidates every embedding you have. They live in different vector spaces, so the old ones become meaningless rather than merely stale. Immich handles it — you re-run Smart Search over the whole library — so make the model decision before you start the long job. I learned this the way you’d expect.

Multilingual is a real consideration if your household doesn’t search in English. The multilingual models are noticeably weaker at English than the English-only ones of the same size, so this is a genuine trade rather than a free upgrade.

Running the job

Administration → Jobs. Smart Search → All, Face Detection → All. Then watch:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

$ watch -n5 nvidia-smi
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|=========================================================================================|
|    0   N/A  N/A     18293      C   python3                                      4212MiB |
+-----------------------------------------------------------------------------------------+
| GPU  Name    Persistence-M | Bus-Id  Disp.A | Volatile Uncorr. ECC                       |
|  0   RTX 3060 Ti       Off | 00000000:01:00.0 Off |                  N/A                 |
| 62%   71C    P2      178W / 200W |   4212MiB /  8192MiB |     94%      Default            |
+-----------------------------------------------------------------------------------------+

94% utilisation is what you want to see. If it’s bouncing between 20% and 60%, the GPU is starved and the bottleneck is elsewhere — usually the server reading original files off spinning disks, or the network if your originals are on a NAS mounted over NFS.

The numbers from my run, 80,000 photos:

Face detection is unchanged by the CLIP model choice, since it uses a separate detector. And the honest note on the CPU column: those figures are Immich’s own estimates extrapolated from the first hour. I didn’t have nine days of curiosity.

The database half nobody mentions

Embeddings are useless without something that can search them, and Immich’s answer is pgvecto.rs — a Rust vector extension for Postgres that the compose file pulls in as tensorchord/pgvecto-rs. It is a hard dependency; a stock Postgres image will start, accept connections, and then fail every migration Immich tries to run.

This matters when you move the GPU work off-box, because it’s the one piece that has to stay next to the server. The vectors are written by the server after the ML container hands back an array of floats, so the network carries images one way and small float arrays back, and the database never leaves home.

Two settings on the Postgres container are worth attention on a large library:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

  database:
    image: tensorchord/pgvecto-rs:pg14-v0.2.0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    command: >-
      postgres
      -c shared_preload_libraries=vectors.so
      -c search_path="$$user", public, vectors
      -c shared_buffers=1GB
      -c maintenance_work_mem=512MB
    volumes:
      - ./pgdata:/var/lib/postgresql/data

maintenance_work_mem is the one that bites. The vector index gets rebuilt after a bulk re-index, and on a default 64 MB budget that rebuild takes hours and produces a worse index. Give it half a gigabyte for the duration of a big job. --data-checksums is unrelated to vectors and is simply what you want on any database holding something you can’t regenerate — an unnoticed bit-flip in an embeddings table produces search results that are wrong in ways you’d never trace back to hardware.

Faces are a different job with different rules

Face recognition runs in two stages that people conflate. Detection finds faces and embeds them — that’s the GPU job, and it’s the slow one. Recognition clusters those embeddings into people, and it’s pure database work that takes minutes rather than hours.

The consequence is practical: the clustering thresholds in Administration → Settings → Machine Learning → Facial Recognition can be re-tuned without touching the GPU. Detection stays valid; only the clustering re-runs.

Max recognition distance is the dial. Lower means stricter, which produces one person split across six “people” entries. Higher means looser, which merges your two siblings into one person and is much more annoying to unpick. The default of 0.5 was slightly too strict for my library, and 0.55 halved the number of duplicate people at the cost of two bad merges I fixed by hand.

Min detected faces — how many instances before Immich shows a person at all — is the setting that decides whether the People page is useful or a wall of strangers from the background of holiday photos. I run 5.

Troubleshooting

The server logs ECONNREFUSED to the ML host. The URL needs a scheme and a port — http://192.168.1.20:3003, with none of it optional. Also check the bind address on the container: 127.0.0.1:3003 in the ports mapping means the machine can reach it and nothing else can.

Jobs run but nvidia-smi shows nothing. You’re on the non-CUDA image, or the container toolkit isn’t configured on the host. docker exec immich-ml nvidia-smi is the direct test — if that fails, the problem is entirely outside Immich.

CUDA failed with error out of memory partway through. Either MACHINE_LEARNING_WORKERS is above 1, or the model you picked wants more VRAM than the card has, or something else grabbed the card mid-run. The 384-pixel SigLIP variants need meaningfully more than the base ones — the input resolution is in the name and it costs what you’d expect.

First job of the day is slow, subsequent ones are fast. MACHINE_LEARNING_MODEL_TTL expired and it’s loading the model from disk again. Working as designed. Set it to 0 if the latency bothers you more than the idle VRAM does.

Search results are worse after changing the model. The library is now a mix of embeddings from two different vector spaces, which produces genuinely random results for anything not yet re-indexed. Re-run Smart Search → All and wait it out.

Model download fails behind a restrictive network. The container pulls from Hugging Face on first use, and the /cache volume is what stops it doing that repeatedly. If the GPU box has no internet, copy a populated cache volume across from a machine that does.

Everything works, then breaks after docker compose pull. Immich is still in heavy development and releases breaking changes with some regularity. The server and ML container versions must match — that’s what the shared IMMICH_VERSION variable is for, and running release on one box and a pinned tag on the other is a mistake that produces baffling errors. Pin both, upgrade both, read the release notes. This is the standing tax on running Immich and it hasn’t changed.

Is it worth it?

If your library is under about 5,000 photos, no. The CPU will finish overnight, you’ll never think about it again, and a second machine adds a failure mode for a job that runs once.

Above roughly 20,000 photos, the GPU changes what’s possible rather than what’s fast. It moves the model choice from “the smallest one you can stand” to “the best one available”, and that shows up in every search you run afterwards. It also means new imports get embedded in seconds instead of minutes, so the library is searchable while you still remember taking the picture.

The split deployment is the part I’d recommend regardless of scale, because it costs nothing. If you already have a machine with a card in it — a gaming desktop, a machine running local models, anything with an eGPU hanging off it — one environment variable borrows it for the twenty minutes a week Immich needs, and hands it straight back.

The thing I’d do differently: choose the model first. I ran the whole library through ViT-B-32, decided the searches were mediocre, switched to SigLIP, and ran the whole library again. Six hours I could have skipped by spending ten minutes reading the model list before I clicked the button.