CS578: Internet of Things What is MQTT?

MQTT : Message Queueing Telemetry Transport protocol.

MQTT: •
•
Reliable, Lightweight, Cost-effective protocol that transports messages between devices.
Suited for the transport of telemetry data (sensor and actuator data)

Message Queuing Telemetry Transport • Invented by Andy Stanford Clark of IBM

and Arlen Nipper of Arcom (now Eurotech)
in 1999

• Used by real-life IoT frameworks:

• Amazon Web Services (AWS),
• IBM WebSphere MQ,
• Microsoft Azure IoT,
• Facebook Messenger,
• etc.

MQTT Architecture
Example:
• Light sensor continuously publish sensor data to the broker.
Dr. Manas Khatua • Building control application subscribes to light sensor data and so receives it from the broker. Then
Assistant Professor, Dept. of CSE, IIT Guwahati it decides to activate Camera.
E-mail: [email protected] • The application sends an activation message to the camera node (i.e. actuator) through the broker.
“Lead us from unreal to real; from darkness into
11-10-2019 light; from
Dr. Manas death to immortality; Om Shanti Shanti Shanti.”
Khatua 1 11-10-2019 Dr. Manas Khatua 2

MQTT Characteristics Advantages of MQTT

• In 2013-14, MQTT was adopted
and published as an official
standard by OASIS
• OASIS: Organization for the
Advancement of Structured
Information Standard

• MQTT was mainly designed for

M2M communication

 Publish / Subscribe (PubSub) model

 Decoupling of data producer (publisher) and data consumer (subscriber) through topics (message queues)

 Asynchronous communication model with messages (events)

 Publisher and Subscriber need not be online at the same time

 Low overhead (2 bytes header)

 Suitable for low network bandwidth applications

 Runs on connection-oriented transport (TCP), but it will be used in conjunction with 6LoWPAN HC
(TCP header compression). Source: https://www.u-blox.com/en/blogs/insights/mqtt-beginners-guide

11-10-2019 Dr. Manas Khatua 3 11-10-2019 Dr. Manas Khatua 4

 Publish & Subscribe Messaging Cont…
 A producer sends (publishes) a message (publication) on a topic (subject) Wildcards
 A consumer subscribes (makes a subscription) for messages on a topic (subject)  A subscriber can subscribe to an absolute topic OR can use wildcards:
 A message server / broker matches publications to subscriptions
 Single-level wildcards “+” can appear anywhere in the topic string
For example:
Who will get the message ? Let the Topic: <country>/<region>/<town>/<postalcode>/<house no>/energyConsumption
 If no matches, the message is discarded
 If one / more matches, the message is delivered to each matching subscriber/consumer  Energy consumption for 1 house in Hursley
 UK/Hants/Hursley/SO212JN/+/energyConsumption

 Energy consumption for all houses in Hursley

Topic  UK/Hants/Hursley/+/+/energyConsumption
 A topic forms the namespace in hierarchical with each “sub topic” separated by /
(forward slash)  Multi-level wildcards “#” must appear at the end of the string
 An example topic space : For example:
 A house publishes information about itself on:  Details of energy consumption, solar and alarm for all houses in SO212JN
<country>/<region>/<town>/<postalcode>/<house no>/energyConsumption  UK/Hants/Hursley/SO212JN/#
<country>/<region>/<town>/<postalcode>/<house no>/solarEnergy
NOTE :
• It subscribes for control commands:  Wildcards must be next to a separator
<country>/<region>/<town>/<postalcode>/<house no>/thermostat/setTemp  Wildcards cannot be used when publishing

11-10-2019 Dr. Manas Khatua 5 11-10-2019 Dr. Manas Khatua 6

Cont… MQTT Message Format

 A subscription can be durable or non-durable
 Durable:
 Once a subscription is in place,
 A broker will forward matching messages to the subscriber immediately if the subscriber is
connected.
 If the subscriber is not connected, messages are stored on the server/broker until the next
time the subscriber connects.

• Message Type: identifies the kind of MQTT packet within a message

• DUP: Duplicate flag – indicates whether the packet has been sent previously or not

• QoS Level: it allows to select different QoS level

• Retain: this flag notifies the server to hold onto the last received PUBLISH message data
• Remaining Length: specifies the size of optional fields

 Non-durable / Transient (i.e. subscription ends with client session):  MQTT is lightweight
• The subscription lifetime is the same as the time the subscriber is connected to the server /  because each packets consists of a 2-byte fixed header with optional variable header and
broker payload fields

11-10-2019 Dr. Manas Khatua 7 11-10-2019 Dr. Manas Khatua 8

 Cont… Message Types
Message header field Description / Values
Message Type (4 bits) 0: Reserved 8: SUBSCRIBE
1: CONNECT 9: SUBACK
2: CONNACK 10: UNSUBSCRIBE
14 message types, 3: PUBLISH 11: UNSUBACK
2 are reserved 4: PUBACK (Publish ACK) 12: PINGREQ
5: PUBREC (Publish Received) 13: PINGRESP
6: PUBREL (Publish Release) 14: DISCONNECT
7: PUBCOMP (Publish Complete) 15: Reserved
DUP (1 bit) Duplicate message flag. Indicates to the receiver that this message may have already been received.
1: Client or server (broker) re-delivers a PUBLISH, PUBREL, SUBSCRIBE or UNSUBSCRIBE message
(duplicate message).

QoS Level (2 bits) Indicates the level of delivery assurance of a PUBLISH message.
0: At-most-once delivery, no guarantees, «Fire and Forget».
1: At-least-once delivery, acknowledged delivery.
2: Exactly-once delivery.

RETAIN (1 bit) 1: Instructs the server to retain the last received PUBLISH message and deliver it as a first message to
new subscriptions.

Remaining Length (1-4 bytes) Indicates the number of remaining bytes in the message, i.e. the length of the (optional) variable length
header and (optional) payload.

11-10-2019 Dr. Manas Khatua 9

RETAIN Remaining Length (RL)

RETAIN=1 in a PUBLISH message instructs the server to keep the message for this topic. • The remaining length field encodes the sum of the lengths of:
When a new client subscribes to the topic, the server sends the retained message quickly. 1. (Optional) variable length header
2. (Optional) variable length payload
 Typical application scenarios:
 Clients publish only changes in data, so subscribers receive the last known good value.
• To save bits, RL is a variable length field with 1 to 4 bytes.
 Example: • The most significant bit (msb) of a length field byte has the meaning continuation bit (CB).
 Subscribers receive last known temperature value from the temperature data topic. • If more bytes follow, it is set to 1.
 RETAIN=1 indicates to subscriber B that the message may have been published some time ago.
RL is encoded as:: a * 1280 + b * 1281 + c * 1282 + d * 1283
and placed into the RL field bytes as follows:
7 bits
msb

11-10-2019 Dr. Manas Khatua 11 11-10-2019 Dr. Manas Khatua 12

 MQTT QoS Cont…
 Even though TCP/IP provides guaranteed data delivery, data loss can still occur if a TCP connection QoS level 0:
breaks down and messages in transit are lost.  At-most-once delivery («best effort»).
 Messages are delivered according to the delivery guarantees of the underlying network (TCP/IP).
 Therefore, MQTT adds 3 quality of service (QoS) levels on top of TCP  Example application: Temperature sensor data which is regularly published. Loss of an individual
value is not critical since applications (i.e. consumers of the data) will anyway integrate the values
over time

QoS level 1:
• At-least-once delivery.
 Messages are guaranteed to arrive, but there may be duplicates.
 Example application: A door sensor senses the door state. It is important that door state
changes (closed->open, open->closed) are published losslessly to subscribers (e.g. alarming
PUBREC (REC: received) function). Applications simply discard duplicate messages by comparing the message ID field.
Packet is the response to a
PUBLISH Packet
QoS level 2:
at least once
PUBREL (REL: release) Packet  Exactly-once delivery.
is the response to a PUBREC  This is the highest level that also incurs most overhead in terms of control messages and the need
Packet for locally storing the messages.
 Exactly-once is a combination of at-least-once and at-most-once delivery guarantee.
PUBCOMP (COMP: complete)
 Example application: Applications where duplicate events could lead to incorrect actions, e.g.
Packet is the response to a
PUBREL Packet. sounding an alarm as a reaction to an event received by a message. So, it avoids duplicate.

11-10-2019 Dr. Manas Khatua 13 11-10-2019 Dr. Manas Khatua 14

PUBLISH msg flow Cont…

QoS level 0: QoS level 2:
 With QoS level 0, a message is delivered with at-most-once delivery semantics («fire-and-forget»).  QoS level 2 affords the highest quality delivery semantics exactly-once, but comes with the cost of
additional control messages.

QoS level 1:
 QoS level 1 affords at-least-once delivery semantics. If the client does not receive the PUBACK in
time, it re-sends the message.

11-10-2019 Dr. Manas Khatua 15 11-10-2019 Dr. Manas Khatua 16

 Example: CONNECT & SUBSCRIBE msg flow Cont…
Case 1: Case 2:
• Session/connection and subscription setup with clean session flag = 1 (non-durable subscription) • Session/connection and subscription setup with clean session flag = 0 (durable subscription)

11-10-2019 Dr. Manas Khatua 17 11-10-2019 Dr. Manas Khatua 18

What is MQTT-SN? Demo using MQTT Protocol

Send Sensor values to Cloud using MQTT Protocol:
MQTT-SN (MQTT for sensor networks) is a variant of MQTT that has been optimized for
use in low power environments such as sensor networks, as the name suggests.
• The Ultrasonic Sensor
SN adds extra functionality for use cases where lower power is required. reads the distance
between itself & the
• QoS mode -1: allows for fire-and-forget messaging object

• Topic aliases: allows for simplified publishing and reduced data overheads • Sends the measured
distance to the cloud
• Sleep mode (disconnected sessions): allows messages to be queued on the broker while the server using MQTT
remote Thing or device is powered off
protocol.

• Here, the cloud is

ThingSpeak.com

11-10-2019 Dr. Manas Khatua 19 11-10-2019 Dr. Manas Khatua 20

Requirements ESP8266 NodeMCU: Libraries
Hardware
•ESP8266 NodeMCU The ESP8266 NodeMCU requires certain libraries for optimal use of the board.
•Breadboard
•Jumper Wires • PubSubClient – It is required for the MQTT Messaging.
•HC-SR04 Ultrasonic Sensor • ThingSpeak – It is a communication library for supporting to send data to the cloud.
•Raspberry Pi • ESP8266WiFi – It is required to use the Wi-Fi capabilities of the NodeMCU.

Software
•Arduino IDE
•Fritzing Step to install above libraries for the programming part:
• Open Arduino IDE
Cloud Platform • Move on to Tools
•ThingSpeak Account • Manage Libraries…
• Search and install the above mentioned libraries
ThingSpeak is an IoT analytics PaaS that
allows you to aggregate, visualize, and
analyze live data streams

11-10-2019 Dr. Manas Khatua 21 11-10-2019 Dr. Manas Khatua 22

Setup Connections Programming ESP8266

#include <ESP8266WiFi.h> void setup_wifi () {
Subscribe to Cloud #include <PubSubClient.h> delay(10);
TP-LINK_ED34 Topic “distance”
Server Serial.println();
#define TRIGGER D6 Serial.print("Connecting to ");
Publish on MQTT Client #define ECHO D8 Serial.println(ssid);
Topic “distance” long duration, cm;
(Subscriber)
WiFi.begin(ssid, password);
const char* ssid = "TP-LINK_ED34"; while (WiFi.status() != WL_CONNECTED) {
const char* password = "48193580"; delay(500);
Serial.print(".");
//Raspberry Pi IP Address }
const char* mqtt_server = "192.168.0.104"; Serial.println("");
Serial.print("WiFi connected - ESP IP address: ");
WiFiClient espClient;
Serial.println(WiFi.localIP());
PubSubClient client(espClient); }
MQTT Client
(Publisher) MQTT Server
(Broker)
"192.168.0.104"

11-10-2019 Dr. Manas Khatua 23 11-10-2019 Dr. Manas Khatua 24

 Cont… Cont…
void setup () { digitalWrite(TRIGGER, LOW);
void callback (String topic, byte* message, void reconnect () delayMicroseconds(2);
unsigned int length) { Serial.begin(115200);
pinMode(TRIGGER,OUTPUT); digitalWrite(TRIGGER, HIGH);
{ // Loop until we're reconnected delayMicroseconds(10);
Serial.print("Message arrived on topic: "); while (! client.connected ()) { pinMode(ECHO,INPUT);
digitalWrite(TRIGGER, LOW);
Serial.print(topic); Serial.print("Attempting MQTT connection...");
Serial.print("Message: "); setup_wifi ();
duration = pulseIn(ECHO, HIGH);
if (client.connect("ESP8266Client"))
String messageTemp; { client.setServer (mqtt_server, 1883);
cm = duration / 29 / 2;
for (int i = 0; i < length; i++) { Serial.println("connected"); static char distanceincm[7];
Serial.print((char)message[i]); client.subscribe("esp8266/4"); client.setCallback (callback);
messageTemp += (char)message[i]; } }
dtostrf(cm, 6, 2, distanceincm);
} else {
Serial.println(); Serial.print("failed, rc="); client.publish("/esp8266/distance", distanceincm);
} Serial.print(client.state());
Serial.println(" try again in 5 seconds"); void loop () { Serial.print("Distance: ");
delay(15000); if (! client.connected ()) { Serial.print(cm);
} reconnect();
A callback is a function that is passed Serial.print("cm");
} } Serial.println();
as an argument to another function. } if(! client.loop() ) delay(5000);
client.connect("ESP8266Client"); }

11-10-2019 Dr. Manas Khatua 25 11-10-2019 Dr. Manas Khatua 26

Programming R. Pi ThingSpeak Cloud Dashboard

import paho.mqtt.client as mqtt
import urllib3 # powerful HTTP client for Python if val != '' :
conn = http.request('GET', baseURL +'&field1=%s'%(val))
myAPI = 'SMLI56456456RHUB' print(conn.status)
baseURL = 'https://api.thingspeak.com/update?api_key=%s' conn.read()
% myAPI conn.close()
val='' val = ''
http = urllib3.PoolManager()
def main():
def on_connect (client, userdata, flags, rc): mqtt_client = mqtt.Client()
print("Connected with result code "+ str(rc)) mqtt_client.on_connect = on_connect
client.subscribe("/esp8266/distance") mqtt_client.on_message = on_message

def on_message (client, userdata, message): mqtt_client.connect ('localhost', 1883, 60)

print("Received message '" + str(message.payload) + # Connect to the MQTT server and process messages in a
"' on topic '" + message.topic) background thread.
distance = 0 mqtt_client.loop_start()
if message.topic == "/esp8266/distance":
print("Distance updated") if __name__ == '__main__':
distance = str(message.payload, 'UTF-8') print('MQTT to InfluxDB bridge')
distance = distance.strip() main()
print(distance)
global val
val = distance

11-10-2019 Dr. Manas Khatua 27 11-10-2019 Dr. Manas Khatua 28

 Thanks!

11-10-2019 Dr. Manas Khatua 29