MQTT Essentials: The Protocol Behind Every Smart Home

If you’ve spent any time poking at a smart home, you’ve bumped into MQTT whether you meant to or not. Zigbee2MQTT, ESPHome, Tasmota, Home Assistant’s own internal bus — they all lean on it. It’s worth understanding properly, because once you do, an awful lot of “magic” stops being magic and starts being a handful of topics and a tiny broker process.

1 What it actually is

MQTT is a publish/subscribe messaging protocol. It was designed in 1999 for oil pipeline telemetry over satellite links, which tells you everything about its priorities: tiny payloads, low overhead, and tolerance for flaky connections. Devices don’t talk to each other directly. They talk to a broker — a small server that receives messages and fans them out to whoever has subscribed.

Messages are organised into topics, which look like file paths:

zigbee2mqtt/kitchen_motion
home/livingroom/temperature
tele/garage_door/STATE

A device publishes a value to a topic. Anything that has subscribed to that topic gets the value pushed to it. The publisher has no idea who’s listening, and frankly doesn’t care. That decoupling is the whole point — your motion sensor doesn’t need to know that three automations, a dashboard, and a logging script all want its data.

2 Running a broker

The de-facto broker is Eclipse Mosquitto. It’s a single C binary with a footprint so small you’ll forget it’s running. Here’s a sane self-hosted setup:

services:
  mosquitto:
    image: eclipse-mosquitto:2
    container_name: mosquitto
    restart: unless-stopped
    ports:
      - "1883:1883"
    volumes:
      - ./config:/mosquitto/config
      - ./data:/mosquitto/data
      - ./log:/mosquitto/log

The one trap newcomers hit: Mosquitto 2.x stopped allowing anonymous connections by default, which is correct but surprises people. You need an explicit config file at ./config/mosquitto.conf:

listener 1883
persistence true
persistence_location /mosquitto/data/
log_dest stdout
password_file /mosquitto/config/passwd

Then create a user:

docker exec -it mosquitto mosquitto_passwd -c /mosquitto/config/passwd homeassistant

Restart the container and you’ve got an authenticated broker. Don’t be the person who exposes port 1883 to the internet with allow_anonymous true — there are bots that scan for exactly that, and an open broker leaks every sensor reading in your house.

3 Watching the traffic

The fastest way to understand MQTT is to subscribe to the firehose and watch. The mosquitto_sub and mosquitto_pub clients are perfect for this:

# Subscribe to everything (the # wildcard matches all levels)
mosquitto_sub -h localhost -u homeassistant -P secret -t '#' -v

zigbee2mqtt/kitchen_motion {"occupancy":true,"battery":87,"linkquality":120}
home/livingroom/temperature 21.4
tele/garage_door/STATE {"POWER":"OFF"}

The -v flag prints the topic alongside the payload, which you want. There are two wildcards: + matches a single level (home/+/temperature), and # matches everything below a point (zigbee2mqtt/#). Publishing is just as direct:

mosquitto_pub -h localhost -u homeassistant -P secret \
  -t 'home/livingroom/setpoint' -m '20.5'

That single line is enough to drive a thermostat if something is subscribed to the topic. This is where MQTT stops being abstract — you can poke your whole house from a shell.

4 QoS, retain, and the bits that bite

Two features cause most of the confusion, so learn them now.

Retain tells the broker to hold onto the last message on a topic and immediately deliver it to any new subscriber. This is why your dashboard shows the current temperature the instant it connects, rather than waiting for the next reading. Sensors usually retain their state; events (like “button pressed”) usually don’t, because re-delivering a stale button press on reconnect is chaos.

QoS (Quality of Service) has three levels: 0 (fire and forget), 1 (delivered at least once, possibly duplicated), and 2 (delivered exactly once, with more handshaking). For home automation, QoS 0 or 1 is almost always right. QoS 2 is rarely worth its overhead unless a duplicated message would actually cause harm.

The classic footgun is a retained message you can’t get rid of. If a misconfigured device retained nonsense to a topic, it haunts you forever. Clear it by publishing an empty retained message:

mosquitto_pub -h localhost -u homeassistant -P secret \
  -t 'zigbee2mqtt/old_device' -r -n

The -r retains, the -n sends a null payload, and the broker deletes the retained entry.

5 Is it worth it / who is this for

MQTT is worth understanding for anyone running more than a couple of smart devices, and essential if you self-host Home Assistant with Zigbee or ESPHome gear. It is genuinely simple — there’s no sprawling spec to wade through, and an afternoon with mosquitto_sub teaches you most of what matters.

It’s overkill if your “smart home” is two cloud-locked plugs controlled by an app; you’ll never see the broker. But the moment you want local control, automations that survive an internet outage, and the freedom to wire any device to any logic, MQTT is the layer everything sits on. Run a broker, subscribe to # for an evening, and watch your house narrate itself. It’s the most useful hour you’ll spend in home automation.