Paperless + a Local LLM: Auto-Tagging Scans Offline

Paperless-ngx solved the boring half of my paperwork years ago. I feed it scans, it runs OCR, and every invoice, letter and warranty becomes full-text searchable in a tidy web interface. What it never solved was the tedious half: putting each document in the right place. A fresh scan lands with no tags, no correspondent, and a filename like scan_0042.pdf, and unless I stop and classify it by hand, my archive slowly rots into a searchable-but-unsorted heap. For a while I did the classifying. Then I got a local language model reading the OCR text and doing it for me, offline, and I stopped touching the pile entirely.

This is a satisfying homelab project because the ingredients are already in the house. If you run Paperless-ngx you have documents with clean OCR text and a proper REST API. If you’ve dabbled in local AI you have Ollama and a small model. The work is a loop that connects the two: fetch the untagged documents, ask the model what they are, write the answer back. Because both halves run on your own hardware, the most sensitive documents you own — bank statements, medical letters, contracts — get read and sorted by software that has no way to send them anywhere.

Why not just use the built-in classifier

Paperless-ngx already ships automatic tagging. Its matching algorithms include an “auto” mode that trains a classifier on the documents you’ve tagged by hand and then predicts tags for new ones. It works, and for a while it was all I used. Two things pushed me past it.

First, it needs a training set. The classifier learns from your existing labels, so a brand-new tag with two example documents behind it predicts badly until you’ve hand-tagged a dozen more. The system is only as good as the filing you were trying to avoid doing.

Second, it can’t reason about content it hasn’t seen a pattern for. A statistical classifier matches surface features; it has no idea that “the enclosed figures relate to the period ending 5 April” means this is a tax document. A language model does, because it read the sentence and understood it. That zero-shot ability is the whole reason to reach for an LLM: you hand it a list of your tags and a document it has never seen, and it picks sensibly on the first try with no training data at all.

The trade is honesty about cost and speed. The classifier is instant and runs on a potato. The LLM takes a few seconds per document and wants a bit of GPU to be pleasant. For a home archive that grows by a handful of scans a day, that trade is trivial, and you can run the whole thing as a nightly job while you sleep.

Why local, specifically

You could point any of this at a hosted API and get slightly better classification for a few pence per document. I won’t, and the reason is the documents themselves. A Paperless archive is one of the most concentrated collections of personal data a household owns — years of finances, health, identity and correspondence in one searchable pile. Sending each new scan’s OCR text to a third party to be tagged means streaming exactly that data to a company whose retention policy I can’t audit. Running the model locally means the text never leaves the machine that already stores it, and I can prove that the same way I prove it for every offline service: unplug the network and watch the nightly job still run.

There’s a practical bonus too. A local model has no rate limit and no bill, so I can re-tag the entire back catalogue on a whim — reclassify five thousand documents against a revised tag scheme overnight — without watching a meter. That kind of bulk reprocessing is the sort of thing you only do when it’s free.

The shape of the integration

Paperless-ngx exposes everything over a token-authenticated REST API, which is what makes this clean. The loop is four steps:

1. Ask Paperless for documents that have no tags yet.
2. For each one, read its OCR content field.
3. Send that text to the local model with the list of allowed tags and ask it to choose.
4. PATCH the chosen tag IDs back onto the document.

Tags in Paperless are objects with numeric IDs, so the model can’t just emit free text — it has to pick from your actual tag list, and you translate its choices into IDs before writing them back. Constraining the model to a fixed menu is the single most important design decision here; let it invent tag names and you get a hundred near-duplicate tags and a worse mess than you started with.

Ollama makes the model half easy. Pull a capable small model and ask for JSON output so you get something parseable instead of a chatty paragraph:

1

docker compose exec ollama ollama pull llama3.1:8b

Here is the core of the tagging loop. It’s deliberately plain Python against the two HTTP APIs so you can see every step rather than trusting a black box with your bank statements:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

import requests

PAPERLESS = "http://paperless.mylab.local:8000"
OLLAMA = "http://ollama.mylab.local:11434"
TOKEN = "your-paperless-api-token"       # from the Paperless admin, per-user
H = {"Authorization": f"Token {TOKEN}"}

# Build a name -> id map of the tags you already created in Paperless.
tags = {}
url = f"{PAPERLESS}/api/tags/?page_size=200"
while url:
    r = requests.get(url, headers=H).json()
    tags.update({t["name"]: t["id"] for t in r["results"]})
    url = r["next"]

allowed = ", ".join(tags)

def classify(text):
    prompt = (
        "You are filing a scanned document. Choose the tags that apply "
        f"from this exact list and no others: {allowed}. "
        "Reply with JSON like {\"tags\": [\"Invoice\", \"Utilities\"]}.\n\n"
        f"Document text:\n{text[:4000]}"
    )
    r = requests.post(f"{OLLAMA}/api/generate", json={
        "model": "llama3.1:8b",
        "prompt": prompt,
        "format": "json",     # forces valid JSON out of Ollama
        "stream": False,
    }).json()
    import json
    chosen = json.loads(r["response"]).get("tags", [])
    return [tags[name] for name in chosen if name in tags]

# Find untagged documents and tag them.
docs = requests.get(
    f"{PAPERLESS}/api/documents/?tags__isnull=true&page_size=25",
    headers=H,
).json()["results"]

for doc in docs:
    ids = classify(doc["content"])
    if ids:
        requests.patch(f"{PAPERLESS}/api/documents/{doc['id']}/",
                       headers=H, json={"tags": ids})
        print(f"#{doc['id']} '{doc['title']}' -> {ids}")

That truncation to the first 4,000 characters is deliberate. The opening of a document — letterhead, subject line, sender, date — carries almost all the classification signal, and feeding the entire OCR text of a twelve-page contract wastes context window and time for no accuracy gain. Send the top of the page and move on.

Drop this into a container or a cron job on the Paperless host, run it nightly, and the untagged pile drains itself while you’re asleep. The same pattern extends to correspondents and document types — they’re separate API endpoints with the same ID-based shape — once you trust the tagging.

Getting good classifications instead of confident nonsense

The quality of this system lives in the prompt and the tag list, and both reward a little discipline.

Give the model a closed menu. The instruction to choose “from this exact list and no others” is doing real work; without it, a model handed a gas bill will cheerfully invent a tag called “Energy Provider Correspondence” that overlaps three tags you already have. Constrain it hard and post-filter its answer against your real tag names, exactly as the script does with if name in tags.

Keep tags coarse and orthogonal. A model picks confidently between “Invoice”, “Statement” and “Letter”; it flounders between forty hair-splitting sub-categories, and so would you. If your tag scheme is a mess, the LLM will faithfully reproduce the mess.

Feed it structure when you have it. If your scanner names files by source, or your consume folder sorts by inbox, pass that hint in the prompt. A model told “this arrived in the utilities inbox” tags utilities documents almost perfectly.

And use format: "json". Ollama’s JSON mode constrains the output to valid JSON, which turns “parse the model’s rambling reply” from a fragile exercise in string-hunting into a single json.loads. It’s the difference between a script that runs unattended for months and one that falls over the first time the model gets chatty.

Troubleshooting: where the loop breaks

The model returns tags that don’t exist. Expected, and already handled — the if name in tags filter silently drops hallucinated names. If it happens a lot, your prompt isn’t constraining hard enough or your tag names are ambiguous. Restate the closed-list instruction and simplify the names.

Everything gets tagged, including junk. A pushy model will slap “Important” on a pizza menu. Add an explicit escape hatch to the prompt — “reply with an empty list if nothing clearly applies” — and the model will leave genuinely ambiguous documents untagged for you to handle by hand, which is the correct behaviour.

json.loads throws. Either you forgot format: "json", or the model wrapped its JSON in a markdown code fence. Turning on JSON mode fixes the first; if a stubborn model still wraps output, strip everything before the first { and after the last } before parsing.

It tags the same documents every run. Your query isn’t excluding already-processed ones. tags__isnull=true only finds documents with zero tags, so anything you tagged last night is correctly skipped — but if you’re testing against documents that already have one tag, widen or narrow the filter to match what you actually mean by “needs tagging”.

Classification is slow and pegs the CPU. The model fell back to CPU because it didn’t fit in VRAM or the GPU wasn’t wired into the container. Confirm with nvidia-smi that VRAM is in use during a run; if not, shrink to a smaller quantised model or fix the GPU passthrough. A nightly batch job hides a lot of slowness, so this only bites if you want tagging to feel interactive.

A bad prompt change silently degrades everything. Because the job runs unattended, a regression can quietly mis-file a week of documents before you notice. Log every decision the way the script does, and spot-check the log for a few days after any prompt change.

Where this sits in a local-AI archive

This project reuses the exact model runner you’d stand up for chat or document Q&A, so it pairs naturally with the rest of an offline-AI setup. If you’d rather drive the same Ollama backend through a browser than a cron script, Open WebUI gives you a private front end over the identical models. And once your archive is cleanly tagged, the obvious next step is asking it questions in plain English — a local RAG stack for chatting with your own documents turns a well-organised Paperless library into something you can interrogate a sentence at a time, still without a single byte leaving the house.

Is it worth it, and who for

Build this if your Paperless archive is large enough that hand-filing has become a chore you avoid, and if the sensitivity of those documents rules out sending them to a hosted API. The payoff is an archive that files itself overnight and a back catalogue you can re-tag on a whim when your scheme changes.

Skip it if you scan three documents a month — the built-in classifier, or thirty seconds of manual tagging, will serve you better than standing up a model. And skip it if you don’t already run a GPU-capable box for other local-AI work, because spinning one up solely to tag a trickle of scans is a poor trade.

For me it earned its place the first week. The scanner feeds Paperless, Paperless does the OCR, a small model reads the top of each page overnight, and I wake up to a sorted archive I never touched — with the documents, the model, and the decisions all staying on hardware I own.