MQTT

MQTT is a lightweight publish/subscribe messaging protocol designed for small devices and unreliable networks. A temperature sensor, an ESP32 board or a smart bulb can send a few bytes to a central server, and any number of apps can receive them instantly. This lesson explains the moving parts: the broker, topics, the three delivery levels (QoS), the special features such as retained messages and Last Will, and what a packet looks like on the wire.

What Is MQTT?

MQTT (originally short for MQ Telemetry Transport) runs on top of TCP, so delivery of the bytes is reliable and ordered. Its message header can be as small as 2 bytes, which keeps battery and bandwidth use low. You will find it in sensor networks, smart-home systems, vehicle and factory telemetry and mobile notifications. The two versions in everyday use are 3.1.1 and 5.0. Version 5.0 keeps the same ideas and adds features such as reason codes, user properties, message expiry and shared subscriptions.

The Publish/Subscribe Model

In MQTT nobody talks to anybody directly. Every client connects to one central server called the broker. A client that has something to say publishes a message to a named topic. A client that wants to hear about that subject subscribes to the topic, and the broker forwards each matching message to it.

Compare this with HTTP, where a client sends a request and waits for one response from one server. With MQTT the sensor does not know whether zero, one or fifty apps are listening, and a subscriber receives new data the moment it is published without polling. Because both sides only talk to the broker, they can also go offline and come back independently (see persistent sessions below).

Topics and Wildcards

A topic is a text string that works like a path, with levels separated by /, for example home/livingroom/temperature. Topics are case sensitive (Home and home are different) and you do not create them in advance: publishing to a new topic is enough.

A subscription may use two wildcards. Wildcards are allowed only in subscriptions, never in the topic you publish to.

Topics that start with $, such as $SYS/..., are reserved for the broker, which typically uses them to publish its own statistics. A subscription that starts with a wildcard, such as #, typically does not receive them. Good topic design goes from general to specific (building/floor/room/sensor), uses no leading /, and puts one kind of value in each topic.

Quality of Service (QoS)

Networks drop packets and connections break, so MQTT lets you choose how hard it tries to deliver each message. This setting is called QoS and exists for every publish and every subscription.

Retained Messages, Last Will and Sessions

What a Packet Looks Like

Every MQTT message is a control packet. It starts with a fixed header: one byte whose upper four bits are the packet type and lower four bits are flags, followed by the Remaining Length, which says how many bytes come after it. Then, depending on the packet type, come a variable header and a payload.

MQTT 5.0 adds one more packet, AUTH (15), for extended authentication. Brokers listen on TCP port 1883 for plain MQTT and 8883 for MQTT over TLS.

Try It Yourself with Mosquitto

Mosquitto is a free, open-source broker that also ships the command-line clients mosquitto_sub and mosquitto_pub. Install it on your own computer, then open two terminals. In the first one, subscribe to the temperature of every room (-v prints the topic with each message):

mosquitto_sub -h localhost -t "home/+/temperature" -v

In the second one, publish a value:

mosquitto_pub -h localhost -t home/kitchen/temperature -m 24.5

The first terminal prints home/kitchen/temperature 24.5. Two useful variations:

# retained: a subscriber that joins later still gets this value at once
mosquitto_pub -h localhost -t home/kitchen/temperature -m 24.5 -r

# publish with QoS 1
mosquitto_pub -h localhost -t home/kitchen/temperature -m 24.5 -q 1

Mosquitto 2.0 and later accept connections only from the same computer until you configure a listener, which is fine for this experiment. A graphical client such as MQTT Explorer shows the whole topic tree live and is handy for debugging. Public test brokers exist on the internet, but anyone can read what you send to them, so use them only for throwaway data and never for anything private.

MQTT on the ESP32

On an ESP32 the usual choice is the PubSubClient library (install it from the Arduino IDE Library Manager). It sits on top of a normal Wi-Fi client, so first make sure your board can join your network as shown in Connecting to Wi-Fi. The sketch below connects, subscribes to a command topic and publishes a reading every five seconds.

#include <WiFi.h>
#include <PubSubClient.h>

const char* ssid = "YOUR_WIFI_NAME";
const char* password = "YOUR_WIFI_PASSWORD";
const char* mqttServer = "192.168.1.50";   // your broker's IP address or host name
const int mqttPort = 1883;

WiFiClient espClient;
PubSubClient client(espClient);

unsigned long lastPublish = 0;

// Called for every message that arrives on a topic we subscribed to
void callback(char* topic, byte* payload, unsigned int length) {
  String message;
  for (unsigned int i = 0; i < length; i++) {
    message += (char)payload[i];
  }
  Serial.print(topic);
  Serial.print(": ");
  Serial.println(message);
}

void reconnect() {
  while (!client.connected()) {
    Serial.print("Connecting to MQTT...");
    if (client.connect("esp32-kitchen-01")) {      // client id must be unique
      Serial.println("connected");
      client.subscribe("home/kitchen/led");
    } else {
      Serial.print("failed, rc=");
      Serial.println(client.state());
      delay(2000);
    }
  }
}

void setup() {
  Serial.begin(115200);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println();
  Serial.println(WiFi.localIP());

  client.setServer(mqttServer, mqttPort);
  client.setCallback(callback);
}

void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();                                   // keeps the connection alive

  if (millis() - lastPublish > 5000) {
    lastPublish = millis();
    client.publish("home/kitchen/temperature", "24.5");
  }
}

• client.setServer(host, 1883) and client.setCallback(callback) say where the broker is and which function receives incoming messages.
• client.connect("esp32-kitchen-01") opens the MQTT session with a client id. If your broker needs a login, use the version that also takes a user name and password: client.connect(id, user, pass).
• client.loop() must run often. It receives messages, calls your callback and sends the keep-alive pings.
• PubSubClient is simple on purpose: it typically publishes at QoS 0 and subscribes at QoS 0 or 1, and it has a limit on the size of a message that you can raise in its settings.

Security

• Encrypt with TLS. Use port 8883 so that credentials and data are not readable on the network (the same TLS as in HTTPS). Plain port 1883 sends everything in the clear.
• Authenticate every client with a user name and password, or with client certificates.
• Limit what each client may do with topic access control lists (ACLs) on the broker, for example a sensor may only publish to its own topic.
• Never expose a broker to the internet without authentication. Open brokers are found and abused quickly.

MQTT vs HTTP

They work well together: devices often speak MQTT to the broker, while a web or mobile app reads stored data over HTTP.

Common Mistakes

• A leading slash in the topic. /home/kitchen has an empty first level and is a different topic from home/kitchen. Most designs avoid it.
• Wildcards in a publish topic. + and # work only in subscriptions. Publishing to home/+/temperature is invalid.
• Forgetting client.loop(). Without it an Arduino client never receives messages and the broker drops it when the keep-alive time runs out.
• Two clients with the same client id. The broker allows one connection per id, so the new one kicks the old one off, and they can keep taking each other's place in a loop. Give every device a unique id.
• Using QoS 2 everywhere. It costs four packets per message. Use QoS 0 for frequent readings and QoS 1 for most commands.
• Subscribing to # in production. An app that receives every message on the broker can be overwhelmed. Subscribe only to what you need.