Paperless-ngx OCR Tuning for Awkward Scans

Paperless-ngx works out of the box. You drop a PDF in the consume folder, it OCRs it, tags it, and it’s searchable. For a clean 300 DPI scan from a decent flatbed, the default settings are already the settings I’d recommend and this article can end here.

Then you get to the box of documents that made you install Paperless in the first place. A fax from 2011. A thermal receipt that’s gone brown. A council letter photographed on a phone at an angle. A bank statement whose PDF has an embedded text layer the bank generated, badly, in which every 8 is a B. Those documents are why the settings exist, and the defaults are wrong for all of them.

What actually happens to a document

You can’t tune the pipeline without knowing its shape, and Paperless hides it well enough that people assume it does the OCR itself. It doesn’t. It orchestrates:

1. Paperless receives the file and decides whether it needs OCR at all.
2. OCRmyPDF is invoked with a pile of arguments derived from your environment variables.
3. OCRmyPDF rasterises pages, optionally runs unpaper to clean them, optionally deskews and rotates.
4. Tesseract 5 does the recognition, per page, per language.
5. OCRmyPDF stitches the recognised text back as an invisible layer over the original image and writes a PDF/A.
6. Paperless stores that as the archive version, extracts the plain text for its search index, and runs matching rules.

Every setting below lands somewhere in steps 2–5. And the crucial structural fact: the original file is kept untouched, always. Whatever you break in OCR, you can re-run against the pristine original. This is what makes the whole thing safe to experiment with, and it’s why I’ll tell you to try aggressive settings without much hedging.

The settings that earn their place

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# docker-compose.yml — the webserver service's environment block
    environment:
      PAPERLESS_OCR_LANGUAGE: eng+dan
      PAPERLESS_OCR_MODE: skip
      PAPERLESS_OCR_SKIP_ARCHIVE_FILE: never
      PAPERLESS_OCR_CLEAN: clean
      PAPERLESS_OCR_DESKEW: 'true'
      PAPERLESS_OCR_ROTATE_PAGES: 'true'
      PAPERLESS_OCR_ROTATE_PAGES_THRESHOLD: '12'
      PAPERLESS_OCR_OUTPUT_TYPE: pdfa
      PAPERLESS_OCR_IMAGE_DPI: '300'
      PAPERLESS_OCR_MAX_IMAGE_PIXELS: '256000000'
      PAPERLESS_CONVERT_MEMORY_LIMIT: '512000000'
      PAPERLESS_OCR_USER_ARGS: '{"continue_on_soft_render_error": true, "invalidate_digital_signatures": true}'

Working through the ones that matter.

PAPERLESS_OCR_LANGUAGE: eng+dan. Multiple languages, plus-separated, and the order matters — Tesseract weights the first one. My post is mostly Danish with English bills mixed in, and getting the order wrong measurably degrades both. The language packs must be installed in the image; the official one ships English only, so anything else needs PAPERLESS_OCR_LANGUAGES: dan deu (note: different variable, space-separated, controls installation) which downloads the traineddata at container start.

Adding languages costs time roughly linearly. Three languages means three passes per page. Resist listing every language you can imagine — a document in one language, scanned against a model expecting three, gets worse, because Tesseract’s confidence scoring has more chances to pick a wrong-language word that scores well.

PAPERLESS_OCR_IMAGE_DPI: 300. This is a fallback for images with no DPI metadata, and it is the single highest-leverage setting in the file. Tesseract wants 300 DPI. It works at 200 and struggles. Below 150 it produces confident nonsense. Phone photos carry no meaningful DPI, so this value is what they get assumed at — and the assumption is what Tesseract scales against.

PAPERLESS_OCR_MODE: skip. The big one. Three values, and picking the wrong one is the most common OCR complaint I see:

• skip — if the PDF already has a text layer, use it and don’t OCR. Fast, correct for digitally-generated PDFs.
• redo — throw away the existing text layer, OCR the images, keep the original layout. This is the setting for vendor PDFs with a bad text layer.
• force — rasterise the whole page and OCR it, discarding any vector content. The nuclear option. Makes files larger and text less crisp, and it is the only thing that works on PDFs where the text layer is subtly misaligned with the image.

The default skip is right for 90% of documents and catastrophically wrong for the other 10%, because a PDF with an incorrect text layer looks perfect to Paperless — it has text, so no OCR happens, and the text is wrong. Your searches fail forever and nothing anywhere reports an error. If a document is unfindable and you know it’s in there, this is the first thing to check.

PAPERLESS_OCR_SKIP_ARCHIVE_FILE: never. Always generate the searchable PDF/A. Some people set this to save disk; disk is cheaper than regret, and the archive file is what you’ll hand to an accountant.

Deskew, rotate, and the trap in clean

PAPERLESS_OCR_DESKEW: true straightens pages that went through the feeder at an angle. Tesseract tolerates maybe two degrees of skew gracefully and degrades noticeably past five. Deskew costs a second or two per page and I’ve never seen it hurt. Leave it on.

PAPERLESS_OCR_ROTATE_PAGES: true fixes pages that are upside down or sideways, using Tesseract’s own orientation detection. The _THRESHOLD is a confidence floor — below it, the page is left alone. The default of 12 is reasonable; lowering it to 6 makes rotation more eager and will occasionally flip a page that was fine, which is worse than the problem it solves. Sparse pages (a receipt with eight words on it) give the orientation detector very little to work with and are exactly where it guesses wrong.

PAPERLESS_OCR_CLEAN: clean runs unpaper to remove speckle, borders and scanner noise before recognition. This is where I’d urge care, because there are two values and they behave very differently:

• clean — unpaper’s output goes to Tesseract, and the original image is kept in the PDF.
• clean-final — unpaper’s output goes to Tesseract and into the final PDF.

clean-final is tempting: the archived document looks tidier. It’s also destructive, since unpaper is a heuristic and occasionally eats a faint signature, a light stamp, or a pencil annotation. On a document you might need to prove something with, that’s a real loss and there’s no undo short of re-processing from the original. I run clean, get the recognition benefit, and keep the pixels my scanner actually saw.

Where clean earns its keep: photocopies, faxes, and anything with a grey background. Where it hurts: dense small text, where unpaper’s despeckling can remove the dot on an i.

Reprocessing without re-scanning

Since the original is kept, changing a setting means you can re-run everything. Per document, from the web UI: select, then Actions → Reprocess. For a bulk fix:

1
2
3
4
5
6

# re-OCR every document tagged 'bad-ocr' with the new settings
docker compose exec -T webserver \
  document_archiver --overwrite --document 1234

# or the whole library, if you've changed something fundamental
docker compose exec -T webserver document_archiver --overwrite

Then rebuild the search index, which doesn’t happen automatically:

1

docker compose exec -T webserver document_index reindex

Full-library reprocessing on 4,000 documents took my mini PC about seven hours with two languages enabled. Do it overnight, and do it after you’ve tested the settings on a handful of representative awkward documents rather than discovering at hour six that clean-final ate your signatures.

Testing a setting properly

The feedback loop that made this tractable was skipping Paperless entirely and driving OCRmyPDF directly on one bad document:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# grab the original out of Paperless
docker compose cp webserver:/usr/src/paperless/media/documents/originals/2019/bank-statement.pdf .

# try a combination
docker run --rm -v "$PWD:/data" jbarlow83/ocrmypdf \
  --redo-ocr \
  --deskew \
  --clean \
  --language eng+dan \
  --tesseract-pagesegmode 1 \
  /data/bank-statement.pdf /data/out.pdf

# read what it actually recognised
pdftotext out.pdf - | head -40

Thirty seconds per iteration instead of a Paperless consume cycle. --tesseract-pagesegmode 1 is worth knowing about: mode 1 is automatic page segmentation with orientation detection, mode 6 assumes a single uniform block of text, and mode 6 is dramatically better on receipts and forms where the automatic segmentation decides a two-column layout is one column and interleaves the text. You can pin it in Paperless via PAPERLESS_OCR_USER_ARGS: '{"tesseract_pagesegmode": 6}', though I’d only do that for a dedicated consume folder rather than globally.

Feeding it a box of paper without going mad

The tuning above assumes documents arrive one at a time. A backlog doesn’t. You put forty pages through the feeder and get one forty-page PDF containing eleven unrelated documents, and no OCR setting on earth will split that for you.

Paperless does it with barcodes:

1
2
3
4
5

      PAPERLESS_CONSUMER_ENABLE_BARCODES: 'true'
      PAPERLESS_CONSUMER_BARCODE_STRING: PATCHT
      PAPERLESS_CONSUMER_ENABLE_ASN_BARCODE: 'true'
      PAPERLESS_CONSUMER_ASN_BARCODE_PREFIX: ASN
      PAPERLESS_CONSUMER_BARCODE_SCANNER: ZXING

Print a page of PATCHT separator barcodes, slip one between each document, feed the whole stack in one pass. Paperless splits on the separators and discards them. The forty-page nightmare becomes eleven documents and the feeder ran once.

The ASN barcode is the other half and it’s the reason I’d bother at all. Stick a printed ASN00123 label on the physical paper before scanning, and Paperless reads it and assigns that archive serial number to the document. The digital record and the sheet in the filing box now point at each other. When someone asks for the original of an invoice, you search, read the ASN, and walk to the box.

PAPERLESS_CONSUMER_BARCODE_SCANNER: ZXING rather than the default PYZBAR is worth trying if detection is unreliable — ZXING is noticeably better at barcodes that are slightly rotated or printed on a tired laser printer, which describes every label I make.

Back up the originals, specifically

One consequence of everything above deserves stating plainly: the whole tuning strategy rests on the originals surviving. Paperless keeps them in media/documents/originals/, and every reprocess reads from there. Lose that directory and your archive is frozen at whatever OCR quality it happens to have, permanently, with no path to improvement.

The database matters too — tags, correspondents, ASNs, the matching rules — and it’s small. A working export is one command:

1

docker compose exec -T webserver document_exporter ../export -z -d

-z zips it, -d deletes files in the export target that no longer exist in Paperless. That produces a self-describing archive with a manifest, which is the thing you want in five years when the container you’re running today no longer exists. Feed that export into an offsite backup that you’ve actually tested restoring, because a document archive is precisely the kind of data whose loss you discover at the worst possible moment.

Troubleshooting

“page already has text, skipping OCR” and the text is wrong. PAPERLESS_OCR_MODE: redo, reprocess. As above, this is the big one.

Documents fail with PriorOcrFoundError. skip mode meeting a document that needs the opposite. Reprocess the individual document with redo.

DigitalSignatureError. Some official PDFs are signed, and OCR invalidates the signature. "invalidate_digital_signatures": true in the user args accepts that trade. The signature is a claim about the file you were sent; you’re keeping the original anyway.

Consumption hangs on one large file, and everything queues behind it. PAPERLESS_CONVERT_MEMORY_LIMIT unset means ImageMagick will use all your RAM on a 600 DPI A3 scan and take the box down. Set it. PAPERLESS_OCR_MAX_IMAGE_PIXELS is the related guard — it exists to stop decompression-bomb PDFs, and the default trips on legitimate large-format scans, which looks like a mysterious failure on exactly one document.

OCR output is empty for a page that clearly has text. Usually a page that’s actually an image at low DPI with PAPERLESS_OCR_IMAGE_DPI unset. Check with pdfimages -list yourfile.pdf — it prints the real pixel dimensions per image, and if a full A4 page is 800 pixels wide, no setting will save it. Re-scan.

Recognition is good but matching rules never fire. Different layer of the stack. The text is in the index; your rules are wrong. document_index reindex first to rule out a stale index, then check the rule’s algorithm — “any word” versus “all words” catches people out.

Everything is slow. OCR is CPU-bound and parallelises per page. PAPERLESS_TASK_WORKERS and PAPERLESS_THREADS_PER_WORKER multiply together, and the product should stay at or under your core count. Two workers with four threads each on a 4-core box means constant contention and a net slowdown.

Is it worth it?

For most libraries, tune three settings and stop: language, IMAGE_DPI at 300, and deskew on. That’s twenty minutes and it covers the ordinary awkwardness of ordinary paper.

The redo mode is the one I’d urge everyone to know about even if they change nothing else, because it addresses a failure that’s invisible. A document with a bad embedded text layer produces no error, no warning, and no search hits. You find out years later when you need the thing. Every other OCR problem announces itself.

Beyond that, the returns drop off fast. I spent an evening on --tesseract-pagesegmode and unpaper filter combinations for a stack of thermal receipts, got maybe a 15% improvement in recognised words, and concluded that the receipts were faded past the point where software was the answer. Some documents are simply gone, and the honest move is to re-scan them at 400 DPI in greyscale rather than to keep turning knobs.

What genuinely raised my hit rate more than any OCR setting: better scanning. 300 DPI greyscale rather than 200 DPI bilevel, flatbed rather than phone, and feeding the paper in straight. The pipeline downstream of that is already good and does more with clean input than any amount of configuration does with bad input. Once the text layer is trustworthy, the interesting work moves up a level — to having a local model read and tag the things, which only works as well as the OCR underneath it.