Frigate NVR With a Coral TPU

I ran Frigate on CPU for about six weeks before I gave in and bought a Coral. Six weeks is roughly how long it takes for the novelty of “my cameras can tell a person from a cat” to be outweighed by the fan noise of a machine that never drops below 70% load, all day, forever, so that it can notice a magpie.

The numbers make the case better than the fan noise does. On a modest desktop CPU, running detection on four camera streams cost me around 250 ms per inference and pegged every core Frigate was allowed to touch. With a Coral USB accelerator plugged into the same machine, the same model runs in 7–9 ms and the CPU load for detection drops to near zero. Thirty times faster stops being an optimisation and becomes a change of category: a dedicated appliance becomes a container that quietly coexists with everything else on the box.

If you have not settled on Frigate yet, the general case for it is covered in Frigate: local NVR with real object detection. This post is about the accelerator, the config that keeps CPU load honest, and the specific ways the Coral will annoy you.

Why the Coral is so much faster

The Coral Edge TPU is an ASIC that does one thing: run 8-bit quantised TensorFlow Lite models. It has no general-purpose instruction set worth speaking of, no floating point, and about 8 MB of on-chip SRAM for the model parameters. In exchange, it does roughly 4 trillion 8-bit operations per second at 2 watts.

The 8 MB matters more than the TOPS figure. If your quantised model fits in that SRAM, inference is a stream of data through silicon with no memory round-trips. If it doesn’t fit, the driver pages parts of the model in over USB for every single frame, and your 8 ms inference becomes 60 ms of mostly waiting. Frigate’s default MobileDet model is chosen precisely because it fits. Larger models you find online often do not, and the symptom is a Coral that is technically working while performing like a bad CPU.

Two form factors are worth your attention. The USB accelerator plugs into anything and is the sensible default. The M.2 or mini-PCIe version goes in a spare slot, avoids USB entirely, and is the right answer if you have the slot free and a machine that stays put. I use the USB one because my Frigate host is a mini PC whose single M.2 slot is doing more valuable work holding the OS.

The stack that actually works

The core insight for Frigate config — the one that saves the most CPU — arrives before the Coral does. It’s this: Frigate should never decode your high-resolution stream unless it’s recording it.

Every IP camera worth buying publishes at least two RTSP streams: a main stream at full resolution, and a sub-stream at something like 640×480. Detection happens on the sub-stream, because the detector downscales to 320×320 anyway and there is nothing to gain from feeding it 4K. Recording uses the main stream, copied straight to disk without ever being decoded. Get this wrong and you will burn a core per camera on ffmpeg decode work that gets thrown away, then blame the Coral for not helping.

Here’s a working compose file and config:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

services:
  frigate:
    container_name: frigate
    image: ghcr.io/blakeblackshear/frigate:0.14.1
    restart: unless-stopped
    privileged: true
    shm_size: "256mb"
    devices:
      - /dev/bus/usb:/dev/bus/usb
      - /dev/dri/renderD128:/dev/dri/renderD128
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - /srv/frigate/media:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554"
    environment:
      FRIGATE_RTSP_PASSWORD: "${FRIGATE_RTSP_PASSWORD}"

And the config itself:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

mqtt:
  host: mqtt.mylab.local
  user: frigate
  password: "{FRIGATE_MQTT_PASSWORD}"

detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  hwaccel_args: preset-vaapi

detect:
  enabled: true
  fps: 5

objects:
  track:
    - person
    - car
    - dog
  filters:
    person:
      min_score: 0.6
      threshold: 0.75

record:
  enabled: true
  retain:
    days: 3
    mode: motion
  alerts:
    retain:
      days: 30

cameras:
  driveway:
    ffmpeg:
      inputs:
        - path: rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.1.40:554/stream2
          roles:
            - detect
        - path: rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.1.40:554/stream1
          roles:
            - record
    detect:
      width: 640
      height: 480
    motion:
      mask:
        - 0,0,0,120,640,120,640,0
    zones:
      gate:
        coordinates: 0,480,0,240,300,220,340,480

The shm_size line catches everyone. Frigate’s default shared memory allocation is 64 MB, which is enough for about two cameras. Run four and you get cryptic ffmpeg crashes that look like network problems. Roughly 50 MB per camera at the detect resolutions above is a safe starting point.

detect: fps: 5 is the other free win. Detection at 5 fps catches everything a human walking past a camera does; running at the camera’s native 20 fps quadruples your inference count to track the same person four times per stride. Frigate does object tracking between detections, so the intermediate frames are covered.

The storage maths nobody does first

The Coral solves the compute problem so completely that people forget the other resource cameras eat. Four cameras at 4 Mbit/s, recording continuously, write about 170 GB a day. That is a 2 TB drive filled in twelve days and a consumer SSD’s write endurance consumed in a couple of years.

Two settings decide your fate. retain.mode: motion throws away the segments where nothing moved, which for a driveway camera at night is 95% of them, and typically cuts the figure by an order of magnitude. Separately, alerts.retain.days keeps the clips around a tracked object for much longer than the continuous footage — 3 days of everything and 30 days of the clips that had a person in them is a sane default, and it is what the config above does.

Put the media directory on spinning rust. Frigate writes sequentially in 10-second segments, which is precisely the workload a cheap hard disk is good at, and the recordings database is tiny. Using an SSD here spends endurance on the one workload that gains nothing from it.

Verifying the Coral is actually doing the work

The first thing to check is whether the device enumerated at all:

1
2

$ lsusb
Bus 002 Device 004: ID 18d1:9302 Google Inc.

That 18d1:9302 is the Coral after its firmware has been loaded. On a cold boot it appears as 1a6e:089a Global Unichip Corp. and switches identity the first time a process opens it. This matters enormously and I will come back to it.

Then look at Frigate’s own numbers, under System metrics in the web UI or via the API:

1
2
3
4
5
6
7
8
9

$ curl -s http://192.168.1.20:5000/api/stats | jq '.detectors, .service.uptime'
{
  "coral": {
    "inference_speed": 8.14,
    "detection_start": 0.0,
    "pid": 291
  }
}
604812

inference_speed around 8 ms means the model is resident in the TPU’s SRAM and everything is healthy. Anything over 25 ms means you are either paging the model over USB every frame or the Coral has silently fallen back. Over 100 ms and you are on CPU without realising it.

Troubleshooting

“No EdgeTPU was detected” and Frigate refuses to start. Check lsusb first. If nothing Google-shaped appears, the device is not passed into the container. The /dev/bus/usb bind plus privileged: true is the blunt instrument that always works; udev rules and specific device paths are tidier and break the moment the Coral re-enumerates under a new device number, which it does on every host reboot. Take the blunt instrument.

It works, then stops working after a reboot, and starts again when you unplug and replug it. This is the identity switch described above. The Coral boots as one USB ID and re-enumerates as another once the driver pushes firmware to it. If your container starts before that has happened, or if the container was passed the specific pre-firmware device node, it will never find it. Passing the whole /dev/bus/usb tree makes this a non-issue.

Inference speed sits at 60–80 ms. Two usual causes. First, the Coral is on a USB 2.0 port. It needs USB 3.0 to move frames fast enough, and a black port with only four pins will bottleneck it. Second, you are running a custom model that overflows the on-chip SRAM. Go back to the shipped MobileDet model and see if the number drops; if it does, that was your answer.

The Coral gets hot enough to be uncomfortable to touch. That is normal and by design — the USB version is a plastic case around an aluminium slug that is the entire heatsink. If it is in a cramped cupboard with no airflow it will thermal throttle, which shows up as inference speed climbing slowly over an hour and never recovering. A short USB 3.0 extension cable that gets it out of the dead air behind the machine fixes it, and gives you somewhere to put it that isn’t hanging off the back of a mini PC under its own weight.

CPU load is still high after all of this. The detector is fine and ffmpeg is the problem. Confirm with docker stats and Frigate’s per-camera CPU breakdown. If a camera’s ffmpeg process is using 80% of a core, it is decoding a stream it should be copying. Check that your record role points at the main stream with no re-encode, and that hardware acceleration is actually engaged — preset-vaapi needs /dev/dri/renderD128 passed through, and it silently does nothing if the device isn’t there. The same integrated GPU handles this workload for several cameras without complaint; the mechanics are much the same as Jellyfin hardware transcoding, and the same permissions traps apply.

Objects get tracked but the labels are wrong. MobileDet is trained on COCO, so it knows about 90 everyday object classes and has firm opinions about which one your wheelie bin is. Restricting objects.track to the handful you care about stops Frigate cluttering the timeline with confident identifications of umbrellas. The min_score and threshold pair is worth understanding: min_score is the floor below which an individual detection is discarded, while threshold is the score a tracked object must reach at least once before Frigate considers it real. Raising threshold cuts false alerts without making the tracking jittery; raising min_score too far makes objects flicker in and out of existence as they cross the frame.

Detections fire constantly on nothing. Motion masks come before object filters. Mask out the sky, the road, the neighbour’s tree, and anything else that moves without mattering. Frigate only runs the detector on regions where motion was detected, so a good mask cuts your inference count by an order of magnitude before the Coral ever gets involved.

Where the cameras should live

A quick word, because it undermines everything else if you get it wrong. Cheap IP cameras are among the least trustworthy devices you can plug into a network. Many phone home by default, several have hardcoded credentials, and firmware updates arrive on the manufacturer’s schedule, which is to say never.

Mine sit on a segment with no route to the internet and no route to anything else I care about. Frigate reaches in to pull RTSP; nothing reaches out. That arrangement is the whole point of the setup described in VLAN segmentation at home, and cameras are the single best argument for doing it.

Verdict

The Coral costs about £60 and buys back an entire machine’s worth of CPU. For four cameras it is close to a no-brainer. For one camera at 5 fps, an Intel iGPU running OpenVINO gets close enough that the stick is hard to justify — Frigate supports that detector and it costs nothing extra.

The awkwardness is real. Google’s attention to the Coral line has been thin for years, availability has been erratic, and the USB re-enumeration behaviour is the kind of thing that only makes sense once you have been bitten by it. If you want a comparison against the other serious local option, Scrypted vs Frigate covers where each one wins.

But it does the job it was bought for, at 2 watts, for years. My Frigate host now idles at 6% CPU with four cameras recording continuously, and the fans have not spun up since February. That was the entire objective.