MQTT Explained for People Who Just Want Sensors

Every MQTT explainer starts with “lightweight publish/subscribe messaging protocol for constrained devices”, which is accurate and tells you nothing you can act on. Here is the version that got me building things.

MQTT is a post office. There is exactly one building — the broker. Anyone can drop a letter in, addressed to a topic. Anyone can tell the post office “give me copies of everything addressed to this topic”. The post office never keeps letters for later, never checks whether anyone collected them, and has no idea who anyone is. Everything else is a detail on top of that.

The consequence that matters: nothing in an MQTT system knows about anything else. Your temperature sensor does not know Home Assistant exists. Home Assistant does not know your sensor exists until a message shows up. Neither cares if the other reboots. That decoupling is the entire value proposition, and it is why a smart home built on MQTT survives changes that a directly-integrated one does not.

Topics: it’s just a path

A topic is a string with slashes in it. It looks like a filesystem path and behaves nothing like one — there is no directory, nothing is created, no permission exists until you invent one. It is just a label on an envelope.

1
2
3
4

home/study/temperature
home/study/humidity
home/hall/motion
home/hall/light/set

Convention, which you should follow because your future self will thank you:

• Specific to general, left to right. home/study/temperature, never temperature/study/home. This makes wildcards useful.
• Lowercase, no spaces, no leading slash. A leading slash creates an empty first level and everything downstream gets fiddly.
• Separate state from commands. home/hall/light carries what the light is; home/hall/light/set carries what you want it to be. Publishing a command onto the state topic makes feedback loops trivially easy to build by accident.

Subscribers can use two wildcards. + matches exactly one level: home/+/temperature gets every room’s temperature. # matches everything below and can only appear at the end: home/study/# gets everything in the study. # on its own gets the entire broker, which is the single most useful debugging command there is.

The three settings that matter

Everything else in MQTT is optional. These three decide whether your system works.

QoS

Quality of Service, 0 to 2.

QoS 0 — fire and forget. The publisher sends and never checks. If the network drops the packet, it is gone. For a temperature sensor reporting every 60 seconds, this is correct: a missed reading is replaced by a fresh one a minute later.

QoS 1 — at least once. The broker acknowledges; the publisher retries until it gets the ack. Duplicates are possible if the ack goes missing. For a door sensor, this is correct: you want the event even if the network hiccuped, and a duplicate “door opened” is harmless.

QoS 2 — exactly once. A four-way handshake guaranteeing no duplicates. It is slow, stateful on both ends, and almost never necessary. Use it for a command whose repetition is destructive — “increment the counter”, “unlock the door”. For sensors, never.

The default is QoS 0 and people escalate to 2 as a reflex when things feel unreliable. It rarely helps and it quadruples the traffic.

Retain

This one changes everything and it is the one people miss.

By default the broker forgets a message the instant it forwards it. If your temperature sensor publishes at 10:00 and Home Assistant restarts at 10:00:30, Home Assistant sees nothing until 10:01. For a minute, your dashboard says “unknown”.

A retained message is different: the broker keeps the last one on each topic and hands it to any new subscriber immediately on subscribe. Home Assistant restarts, subscribes, and instantly has the last known value for every sensor.

State topics should be retained. Command topics must never be retained — a retained command means every reconnect replays it, so your light turns on every time the broker restarts, forever, and you will spend an evening hunting a ghost.

Retained messages are sticky in a way that surprises people. Delete the device, and its retained message stays on the broker until something publishes an empty payload to that topic. That is how you clear it:

1
2

$ mosquitto_pub -h 192.168.1.30 -u smarc -P "$MQTT_PASS" \
    -t "home/study/temperature" -r -n

-r retain, -n null message. The topic is now genuinely gone.

Last Will and Testament

The LWT is a message you hand the broker at connection time with the instruction: “if I disappear without saying goodbye, publish this on my behalf.”

1
2
3

$ mosquitto_sub -h 192.168.1.30 -u smarc -P "$MQTT_PASS" \
    -t 'home/#' -v \
    --will-topic 'home/monitor/status' --will-payload 'offline' --will-retain

Without an LWT, a dead device looks exactly like a device that has nothing to say. Your dashboard shows the last temperature it ever reported, forever, and it looks completely plausible. With an LWT on a retained status topic, the broker announces the death within the keepalive timeout and everything downstream can mark it unavailable.

Every device you build should have one. This is the difference between a monitoring system and a system that displays reassuring lies.

Payloads: JSON or plain?

The protocol has an opinion here and it is “I don’t care” — a payload is an opaque byte string up to 256 MB, and the broker never looks inside it. So the choice is yours, and both camps are loud.

Plain values (21.4 on home/study/temperature) are trivial for anything to consume. mosquitto_sub prints them readably, a shell script can use one without a JSON parser, and Home Assistant’s MQTT sensor needs no value_template. The cost is a topic per measurement, so a device with six readings publishes six messages.

A JSON object ({"temperature":21.4,"humidity":44.1,"battery":98} on home/study/state) is one message, one retained value, and one atomic view of the device — every field is from the same instant. Zigbee2MQTT does this and it is the right call for a device whose readings are taken together. The cost is that every consumer needs a template to dig a value out.

My rule: JSON when the fields are taken together and belong together; separate topics when consumers genuinely care about different things. A multisensor publishes one JSON state object. A power meter whose wattage is consumed by four different things and whose daily total is consumed by one gets separate topics, because otherwise every wattage update also re-sends the daily total and every subscriber to either wakes up for both.

Keep payload sizes small regardless. A device publishing a 4 KB JSON blob every second on a shared Wi-Fi segment is a device you will eventually curse.

A broker in five minutes

Mosquitto. It is small, it has been maintained for years, and it does one thing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

services:
  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto:2.0
    restart: unless-stopped
    volumes:
      - ./config:/mosquitto/config
      - ./data:/mosquitto/data
      - ./log:/mosquitto/log
    ports:
      - "1883:1883"
      - "8883:8883"

And config/mosquitto.conf:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

persistence true
persistence_location /mosquitto/data/
log_dest stdout
log_type error
log_type warning
log_type notice

# No anonymous access. Ever.
allow_anonymous false
password_file /mosquitto/config/passwd
acl_file /mosquitto/config/acl

listener 1883
protocol mqtt

listener 8883
protocol mqtt
cafile /mosquitto/config/certs/ca.crt
certfile /mosquitto/config/certs/server.crt
keyfile /mosquitto/config/certs/server.key

Mosquitto 2.0 refuses anonymous connections by default, which caused a great deal of confusion when it landed and was entirely the right call. Create the password file before you start it:

1
2
3
4
5
6

$ docker run -it --rm -v "$PWD/config:/mosquitto/config" \
    eclipse-mosquitto:2.0 mosquitto_passwd -c /mosquitto/config/passwd homeassistant
Password:
Reenter password:
$ docker run -it --rm -v "$PWD/config:/mosquitto/config" \
    eclipse-mosquitto:2.0 mosquitto_passwd -b /mosquitto/config/passwd sensor01 "$SENSOR_PASS"

Give each device its own credentials. It takes ten extra seconds and means a compromised sensor cannot publish as your lock. Then constrain them in config/acl:

1
2
3
4
5
6
7
8
9

user homeassistant
topic readwrite home/#
topic readwrite homeassistant/#

user sensor01
topic write home/study/#
topic read home/study/+/set

pattern write home/%u/status

sensor01 can now publish its own readings and read commands aimed at it, and nothing else. That pattern line with %u substitutes the username, so every device gets a status topic without you writing a rule per device. Persistence keeps retained messages across a broker restart, which you want.

Seeing what is actually happening

This is the command. Learn it and most MQTT debugging becomes trivial:

1
2
3
4
5
6

$ mosquitto_sub -h 192.168.1.30 -u smarc -P "$MQTT_PASS" -t '#' -v
home/study/temperature 21.4
home/study/humidity 44.1
zigbee2mqtt/hall_motion {"battery":98,"occupancy":true,"linkquality":102}
homeassistant/sensor/study_temp/config {"name":"Study Temperature","state_topic":"home/study/temperature",...}
home/study/status online

Every message on the broker, live, with topics. When a sensor “isn’t working”, this tells you within five seconds whether the device is publishing at all, whether it is publishing to the topic you think, and whether the payload is the shape you expect. Roughly three quarters of MQTT problems are one of those three and all of them are visible here.

That homeassistant/sensor/.../config line is MQTT Discovery — a convention where a device publishes a retained JSON description of itself to a well-known topic and Home Assistant creates the entity automatically. It is why Zigbee2MQTT devices appear with no configuration at all. If you build your own sensors with ESPHome, the native API skips MQTT entirely and gets the same result; the trade-off between the two paths is in ESPHome: rolling your own sensors for pennies.

Troubleshooting

Connection refused, and nothing in the logs. Mosquitto 2.0 with allow_anonymous false and no password file will start, listen, and reject everyone. Check that the password file path in the config matches where you actually created it, and that the file is readable by the container user — it runs as UID 1883, and a passwd file created by root with mode 600 is invisible to it.

A sensor connects and immediately disconnects, in a loop. Two devices sharing a client ID. MQTT client IDs must be unique across the broker; when a second client connects with an ID already in use, the broker disconnects the first. The first reconnects, kicking the second, forever. This is common when you flash the same firmware to two boards and forget to change the name.

Home Assistant shows the sensor as unavailable after a restart. No retained flag on the state topic. Publish with -r, or set retain: true in the publishing device’s config.

A light turns itself on whenever the broker restarts. A retained message on a command topic. Clear it with the empty-payload trick above and fix the publisher.

Messages arrive twice. Overlapping subscriptions — a client subscribed to both home/# and home/study/temperature gets two copies. The MQTT spec allows brokers to deliver once per matching subscription, and Mosquitto does.

A device drops off every couple of minutes and reconnects. Keepalive. The client tells the broker “expect to hear from me every N seconds”; if the broker hears nothing for 1.5× that, it declares the client dead and fires the LWT. A device on a marginal Wi-Fi link, or one doing deep sleep for longer than its keepalive, will flap. Raise the keepalive to comfortably exceed the sleep interval, or use clean_session: false so the broker holds the subscription and queues QoS 1 messages while the device naps.

Everything works locally and nothing works from outside. Good. An MQTT broker on port 1883 has no business facing the internet. If a remote device genuinely needs to publish, put it on the mesh VPN and let it reach the broker over that — the arrangement in Tailscale: a zero-config mesh VPN is exactly this, and it beats port forwarding in every dimension.

Verdict

MQTT is worth it the moment you have more than one thing that needs to know about a sensor, and it is worth it immediately if you build your own devices. The protocol is genuinely small — you have now read most of what it does — and the decoupling means you can replace any component without touching the others.

It is unnecessary if everything you own is a Zigbee device and Home Assistant is your only consumer. Zigbee2MQTT publishes to a broker, and if Home Assistant is the sole subscriber, the broker is a hop that buys you nothing. Zigbee2MQTT’s own trade-offs against the direct integration are covered in Zigbee2MQTT vs ZHA.

The moment a second consumer appears — a Node-RED flow, a script, a display in the kitchen, a logger — the broker earns its container immediately and permanently. Mine has been up for years, uses 8 MB of RAM, and is the least interesting service in the rack. For infrastructure, that is the highest compliment available.