Get to Know MQTT: The Messaging Protocol for the Internet of Things

MQTT (formerly the MQ Telemetry Transport) is a lightweight protocol that’s primarily designed for connecting power-constrained devices over low-bandwidth networks. Though it existed for over a decade, the advent of M2M (machine to machine communications) and Internet of Things (IoT) made it a popular protocol.

Developers aspiring to build IoT solutions need to learn MQTT, which is quickly becoming the most preferred protocol for connecting devices to the cloud. Enterprise cloud platforms such as Amazon Web Services, Microsoft Azure, and IBM Watson expose their IoT PaaS through MQTT.

This article introduces the core concepts of MQTT that are necessary for building M2M and IoT applications. It covers the basic terminology to advanced concepts, along with a getting started section to see MQTT in action.

👁 M2M

Source: Eurotech

Origin of MQTT

MQTT was created way back in 1999 by two engineers — Andy Stanford-Clark (IBM) and Arlen Nipper (Eurotech). They had to invent a new protocol for connecting oil pipelines over unreliable, satellite networks.

The motivation for designing MQTT was to create a lightweight and bandwidth-efficient protocol that was data agnostic with support for multiple levels of Quality of Service (QoS). Interestingly, even today, those are the same reasons for which MQTT is chosen for implementing IoT solutions.

In 2011, IBM and Eurotech donated MQTT to the proposed Eclipse project called Paho. In 2013, it was submitted to OASIS for standardization. The latest version of the protocol specification, 3.11 has become an OASIS standard.

Concepts Terminology

Developers familiar with Message Oriented Middleware (MOM) such as RabbitMQ, Redis, and Apache Qpid may mistake MQTT as yet another implementation of MOM. It’s important to understand that MQTT is just a protocol which has nothing to do with message queues or pub/sub implementations. MQTT cannot replace RabbitMQ or any of the enterprise pub/sub engines.

The fundamental difference between MQTT implementation and MOM is the messages are stored and delivered. Unlike MOM, MQTT is not meant for dealing with durable and persistent messages. It cannot be considered for implementing the store-and-forward pattern. While traditional MOM is designed for reliable delivery of messages among enterprise applications, MQTT is a simpler protocol with just five APIs designed to connect devices. It cannot be used as the middleware for transactional systems.

MQTT uses the pub/sub pattern to connect interested parties with each other. It does it by decoupling the sender (publisher) with the receiver (subscriber). The publisher sends a message to a central topic which has multiple subscribers waiting to receive the message. The publishers and subscribers are autonomous, which means that they do not need to know the presence of each other.

👁 Pub-Sub

Since MQTT is a specification, it can be supported by existing MOM software. RabbitMQ supports MQTT among other protocols such as HTTP and AMQP. Azure IoT Hub is an example of how an IoT platform supports MQTT.

Let’s understand the terminology of MQTT.

• Client – Any publisher or subscriber that connects to the centralized broker over a network is considered to be the client. It’s important to note that there are servers and clients in MQTT. Both publishers and subscribers are called as clients since they connect to the centralized service. Clients can be persistent or transient. Persistent clients maintain a session with the broker while transient clients are not tracked by the broker. Clients often connect to the broker through libraries and SDKs. There are over a dozen libraries available for C, C++, Go, Java, C#, PHP, Python, Node.js, and Arduino.

• Broker – The broker is the software that receives all the messages from the publishing clients and sends them to the subscribing clients. It holds the connection to persistent clients. Since the broker can become the bottleneck or result in a single point of failure, it is often clustered for scalability and reliability. It is up to the implementors to decide how to create a scalable broker layer. Some of the commercial implementations of MQTT brokers include HiveMQ, Xively, AWS IoT, and Loop.

• Topic – A topic in MQTT is an endpoint to that the clients connect. It acts as the central distribution hub for publishing and subscribing messages. In typical MOM, a topic is created before the publisher and subscriber connect to the endpoint. In MQTT, a topic is a well-known location for the publisher and subscriber. It’s created on the fly when either of the clients establish the connection with the broker. Topics are simple, hierarchical strings, encoded in UTF-8, delimited by a forward slash. For example, building1/room1/temperature and building1/room1/humidity are valid topic names. Subscribers can choose to subscribe to a specific topic or all the subtopics through wildcards. A subscription to building1/+/temperature will automatically subscribe to the temperature topic of all the rooms in the building. Similarly, building1/#/ will match all the topics available under building1. Refer to the MQTT specification for more details on the wildcards.

• Connection – MQTT can be utilized by clients based on TCP/IP. The standard port exposed by brokers is 1883, which is not a secure port. Those brokers who support TLS/SSL typically use port 8883. For secure communication, the clients and the broker rely on digital certificates. AWS IoT is one of the secure implementations of MQTT, which requires the clients to use the X.509 certificates.

Hands-on with MQTT

Equipped with the key concepts and terminology, let’s set up the testbed to see MQTT in action. We will install the broker and clients to see the how the messages are published. This will help you understand the essence of MQTT.

Installing the broker

The simplest MQTT broker exists in the form of an open source software called Mosquitto. On a Mac with Home Brew, you can install Mosquitto with a single command – brew install mosquitto. For Windows and Linux, please refer to the official installation guide.

The Mosquitto binary is available in /usr/local/sbin. It also needs a configuration file with the settings such as bind address, port, client expiration period, and maximum connections. The default configuration file available at /usr/local/etc/mosquitto/mosquitto.conf can be used without any modifications.

To launch Mosquitto on a Mac, run the following command:

/usr/local/sbin/mosquitto -c /usr/local/etc/mosquitto/mosquitto.conf

This will launch Mosquitto with the default settings. You should see output similar to the following:

👁 4linescode

Installing the Client

Now that the broker is up and running let’s launch the clients which all act as the publisher and subscribers. Though Mosquitto comes with command line tools for pub/sub, it is fun to use a GUI. Let’s download Java-based MQTT client called MQTT.fx from http://mqttfx.jfx4ee.org.

Create two folders on your Mac called pub and sub and then copy MQTT.fx application from the downloaded DMG image to both the folders. Each copy represents the publisher and subscriber respectively.

👁 smallcode

Launch both the copies of the MQTT.fx application and configure them to talk to the local Mosquitto broker. Click on the gear icon to configure the connection settings. Click the + symbol in the Connection Profiles window to add a profile that connects to the Mosquitto running on the localhost.

👁 screens

Make sure that you use unique client ids for the publisher and subscriber. Connect both the instances of MQTT.fx to the Mosquitto broker. Click on the Subscribe tab of subscriber application, and type test/topic1 before clicking the Subscribe button. In the publisher, add the same topic, test/topic1, and type a message in the text box below it.

👁 screens2

You should see the messages flowing from the publisher to the subscriber. You can launch additional subscribers that listen on the same topic. Finally, switch to the command window to see Mosquitto acknowledging the client events.

👁 code

We have successfully set up the broker, publisher, and subscriber on our local machine. We have also seen how to publish messages to the subscribers.

Summary

MQTT is the most preferred protocol for M2M and IoT applications. Based on the pub/sub pattern, it simplifies the connectivity between devices. This article attempted to introduce you to the basics of MQTT. We will cover the advanced concepts of MQTT including QoS, security, LWT, persistence, and the integration with WebSockets in the upcoming articles.