Module – 3 Application Protocols for IoT

IoT Application Layer Protocols:

 IoT industry is working on new lightweight protocols that are better suited to
large numbers of constrained nodes and networks.

 Two of the most popular protocols are CoAP and MQTT.

 Figure highlights their position in a common IoT protocol stack.

 In Figure, CoAP and MQTT are naturally at the top of this sample IoT stack,
based on an IEEE 802.15.4 mesh network.

Dr. Syed Mustafa, HKBKCE.

164
Module – 3 Application Protocols for IoT
IoT Application Layer Protocols:

Dr. Syed Mustafa, HKBKCE.

165
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 Constrained Application Protocol (CoAP) resulted from the IETF Constrained

RESTful Environments (CoRE) working group’s efforts to develop a generic
framework for resource-oriented applications targeting constrained nodes
and networks.

 The CoAP framework defines simple and flexible ways to manipulate sensors
and actuators for data or device management.

Dr. Syed Mustafa, HKBKCE.

166
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 The IETF CoRE working group has published multiple standards-track

specifications for CoAP, including the following:

1. RFC 6690: Constrained RESTful Environments (CoRE) Link Format

2. RFC 7252: The Constrained Application Protocol (CoAP)

3. RFC 7641:Observing Resources in the Constrained Application Protocol (CoAP)

4. RFC 7959:Block-Wise Transfers in the Constrained Application Protocol (CoAP)

5. RFC 8075: Guidelines for Mapping Implementations: HTTP to the Constrained

Dr. Syed Mustafa, HKBKCE.
Application Protocol (CoAP) 167
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 The CoAP messaging model is primarily designed to facilitate the exchange of

messages over UDP between endpoints, including the secure transport protocol
Datagram Transport Layer Security (DTLS).

 CoAP over Short Message Service (SMS) as defined in Open Mobile Alliance for
Lightweight Machine-to-Machine (LWM2M) for IoT device management is also being
considered.

 From a formatting perspective, a CoAP message is composed of a short fixed length

Header field (4 bytes), a variable-length but mandatory Token field (0–8 bytes),
Options fields if necessary, and the Payload field. 168
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

169
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 CoAP message format is relatively simple and flexible.

 It allows CoAP to deliver low overhead, which is critical for constrained

networks, while also being easy to parse and process for constrained devices.

 Table provides an overview of the various fields of a CoAP message.

Dr. Syed Mustafa, HKBKCE.

170
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

Dr. Syed Mustafa, HKBKCE.

171
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

Dr. Syed Mustafa, HKBKCE.

172
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 CoAP can run over IPv4 or IPv6.

 However, the message fit within a single IP packet and UDP payload to avoid
fragmentation.

 For IPv6, with the default MTU size being 1280 bytes and allowing for no
fragmentation across nodes, the maximum CoAP message size could be up to
1152 bytes, including 1024 bytes for the payload.

 In the case of IPv4, as IP fragmentation implementations should limit to more

conservative values and set the IPv4 Don’t Fragment (DF) bit.
Dr. Syed Mustafa, HKBKCE.
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 CoAP communications across an IoT infrastructure can take various paths.

 Connections can be between devices located on the same or different constrained

networks or between devices and generic Internet or cloud servers, all operating over
IP.

 Proxy mechanisms are also defined, and RFC 7252 details a basic HTTP mapping for
CoAP.

 As both HTTP and CoAP are IP-based protocols, the proxy function can be located
practically anywhere in the network, not necessarily at the border between
Dr. Syed Mustafa, HKBKCE.
174
constrained and non-constrained networks.
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

Dr. Syed Mustafa, HKBKCE.

175
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 Just like HTTP, CoAP is based on the REST architecture, but with a “thing” acting as
both the client and the server.

 Through the exchange of asynchronous messages, a client requests an action via a

method code on a server resource.

 A uniform resource identifier (URI) localized on the server identifies this resource.

 The server responds with a response code that may include a resource representation.

 The CoAP request/response semantics include the methods GET, POST, PUT, and
DELETE. Dr. Syed Mustafa, HKBKCE.
176
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 CoAP URI format is similar to HTTP/HTTPS.

 The coap/coaps URI scheme identifies a resource, including host information and
optional UDP port, as indicated by the host and port parameters in the URI.

Dr. Syed Mustafa, HKBKCE.

177
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 CoAP defines four types of messages:

confirmable, non-confirmable, acknowledgement, and reset.

 Method codes and response codes included in some of these messages make them
carry requests or responses.

 CoAP code, method and response codes, option numbers, and content format have
been assigned by IANA as Constrained RESTful Environments (CoRE) parameters.

Dr. Syed Mustafa, HKBKCE.

178
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 While running over UDP, CoAP offers a reliable transmission of messages when a
CoAP header is marked as “confirmable.”

 In addition, CoAP supports basic congestion control with a default time-out, simple
stop and wait retransmission with exponential back-off mechanism, and detection of
duplicate messages through a message ID.

 If a request or response is tagged as confirmable, the recipient must explicitly either

acknowledge or reject the message, using the same message ID, as shown in Figure.

 If a recipient can’t process a non-confirmable message, a reset message is sent.

Dr. Syed Mustafa, HKBKCE.
179
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

Dr. Syed Mustafa, HKBKCE.

180
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 Figure shows a utility operations center on the left, acting as the CoAP client, with the
CoAP server being a temperature sensor on the right of the figure.

 The communication between the client and server uses a CoAP message ID of 0x47.

 The CoAP Message ID ensures reliability and is used to detect duplicate messages.

 The client sends a GET message to get the temperature from the sensor.

Dr. Syed Mustafa, HKBKCE.

181
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 Notice that the 0x47 message ID is present for this GET message and that the message
is also marked with CON.

 A CON, or confirmable, marking in a CoAP message means the message will be

retransmitted until the recipient sends an acknowledgement (or ACK) with the same
message ID.

 temperature sensor does reply with an ACK message referencing the correct message
ID of 0x47.

Dr. Syed Mustafa, HKBKCE.

182
Module – 3 Application Protocols for IoT
Constrained Application Protocol (CoAP):

 In addition, this ACK message piggybacks a successful response to the GET request
itself.

 This is indicated by the 2.05 response code followed by the requested data.

 CoAP supports data requests sent to a group of devices by leveraging the use of IP
Multicast.

 Implementing IP Multicast with CoAP requires the use of all- CoAP-node multicast
addresses.

 For IPv4 this address is 224.0.1.187, Dr.

and for IPv6
Syed Mustafa, HKBKCE.it is FF0X::FD.
183
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 At the end of the 1990s, engineers from IBM and Arcom (acquired in 2006 by
Eurotech) were looking for a reliable, lightweight, and cost-effective protocol to
monitor and control a large number of sensors and their data from a central server
location, as typically used by the oil and gas industries.

 Their research resulted in the development and implementation of the Message

Queuing Telemetry Transport (MQTT) protocol that is now standardized by the
Organization for the Advancement of Structured Information Standards(OASIS).

Dr. Syed Mustafa, HKBKCE.

184
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 At the end of the 1990s, engineers from IBM and Arcom (acquired in 2006 by
Eurotech) were looking for a reliable, lightweight, and cost-effective protocol to
monitor and control a large number of sensors and their data from a central server
location, as typically used by the oil and gas industries.

 Their research resulted in the development and implementation of the Message

Queuing Telemetry Transport (MQTT) protocol that is now standardized by the
Organization for the Advancement of Structured Information Standards(OASIS).

Dr. Syed Mustafa, HKBKCE.

185
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 Considering the harsh environments in the oil and gas industries, an extremely simple
protocol with only a few options was designed, with considerations for constrained
nodes, unreliable WAN backhaul communications, and bandwidth constraints with
variable latencies.

 These were some of the rationales for the selection of a client/server and
publish/subscribe framework based on the TCP/IP architecture, as shown in Figure.

Dr. Syed Mustafa, HKBKCE.

186
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

Dr. Syed Mustafa, HKBKCE.

187
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 An MQTT client can act as a publisher to send data (or resource information) to an
MQTT server acting as an MQTT message broker.

 In the Figure, the MQTT client on the left side is a temperature (Temp) and relative
humidity (RH) sensor that publishes its Temp/RH data.

 The MQTT server (or message broker) accepts the network connection along with
application messages, such as Temp/RH data, from the publishers.

 It also handles the subscription and unsubscription process and pushes the application
data to MQTT clients acting as subscribers.
Dr. Syed Mustafa, HKBKCE.
188
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 The application on the right side of Figure is an MQTT client that is a subscriber to the
Temp/RH data being generated by the publisher or sensor on the left.

 This model, where subscribers express a desire to receive information from publishers,
is well known.

 A great example is the collaboration and social networking application Twitter.

 With MQTT, clients can subscribe to all data (using a wildcard character) or specific
data from the information tree of a publisher.

Dr. Syed Mustafa, HKBKCE.

189
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 The presence of a message broker in MQTT decouples the data transmission between
clients acting as publishers and subscribers.

 In fact, publishers and subscribers do not even know (or need to know) about each
other.

 A benefit of having this decoupling is that the MQTT message broker ensures that
information can be buffered and cached in case of network failures.

 This also means that publishers and subscribers do not have to be online at the same
time.
Dr. Syed Mustafa, HKBKCE.
190
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 MQTT control packets run over a TCP transport using port 1883.

 TCP ensures an ordered, lossless stream of bytes between the MQTT client and the
MQTT server.

 Optionally, MQTT can be secured using TLS on port 8883, and WebSocket (defined in
RFC 6455) can also be used.

 MQTT is a lightweight protocol because each control packet consists of a 2-byte fixed
header with optional variable header fields and optional payload.

 control packet can contain a payloadDr. up to 256

Syed Mustafa, MB.
HKBKCE.
191
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):
Figure provides an overview of the MQTT message format.

Dr. Syed Mustafa, HKBKCE.

192
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 Compared to the CoAP message format in Figure, MQTT contains a smaller header of
2 bytes compared to 4 bytes for CoAP.

 The first MQTT field in the header is Message Type, which identifies the kind of MQTT
packet within a message.

 Fourteen different types of control packets are specified in MQTT version 3.1.1.

 Each of them has a unique value that is coded into the Message Type field.

Dr. Syed Mustafa, HKBKCE.

193
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):
MQTT Message Types

Dr. Syed Mustafa, HKBKCE.

194
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):
MQTT Message Types

Dr. Syed Mustafa, HKBKCE.

195
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 The next field in the MQTT header is DUP (Duplication Flag).

 This flag, when set, allows the client to notate that the packet has been sent
previously, but an acknowledgement was not received.

 The QoS header field allows for the selection of three different QoS levels.

 The next field is the Retain flag.

 Only found in a PUBLISH message , the Retain flag notifies the server to hold onto the
message data.

Dr. Syed Mustafa, HKBKCE.

196
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 This allows new subscribers to instantly receive the last known value without having
to wait for the next update from the publisher.

 The last mandatory field in the MQTT message header is Remaining Length.

 This field specifies the number of bytes in the MQTT packet following this field.

Dr. Syed Mustafa, HKBKCE.

197
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 MQTT sessions between each client and server consist of four phases:

Session establishment, authentication, data exchange, and session termination

 Each client connecting to a server has a unique client ID, which allows the
identification of the MQTT session between both parties.

 When the server is delivering an application message to more than one client, each
client is treated independently.

Dr. Syed Mustafa, HKBKCE.

198
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 Subscriptions to resources generate SUBSCRIBE/SUBACK control packets, while

unsubscription is performed through the exchange of UNSUBSCRIBE/UNSUBACK
control packets.

 Graceful termination of a connection is done through a DISCONNECT control packet,

which also offers the capability for a client to reconnect by re-sending its client ID to
resume the operations.

Dr. Syed Mustafa, HKBKCE.

199
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 PINGREQ/PINGRESP control packets are used to validate the connections between the
client and server.

 Similar to ICMP pings that are part of IP, they are a sort of keepalive that helps to
maintain and check the TCP session.

 Securing MQTT connections through TLS is considered optional because it calls for
more resources on constrained nodes.

 When TLS is not used, the client sends a clear-text username and password during the
connection initiation. Dr. Syed Mustafa, HKBKCE.
200
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

 The MQTT protocol offers three levels of quality of service (QoS).

 QoS for MQTT is implemented when exchanging application messages with publishers
or subscribers.

 The delivery protocol is symmetric.

 This means the client and server can each take the role of either sender or receiver.

 The delivery protocol is concerned solely with the delivery of an application message
from a single sender to a single receiver.
Dr. Syed Mustafa, HKBKCE.
201
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

These are the three levels of MQTT QoS:

QoS 0:

 This is a best-effort and unacknowledged data service referred to as “at most once”
delivery.

 The publisher sends its message one time to a server, which transmits it once to the
subscribers.

 No response is sent by the receiver, and no retry is performed by the sender.

Dr. Syed Mustafa, HKBKCE.
 The message arrives at the receiver either once or not at all. 202
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

QoS 1:

 This QoS level ensures that the message delivery between the publisher and server
and then between the server and subscribers occurs at least once.

 In PUBLISH and PUBACK packets, a packet identifier is included in the variable header.

 If the message is not acknowledged by a PUBACK packet, it is sent again.

 This level guarantees “at least once” delivery.

Dr. Syed Mustafa, HKBKCE.

203
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

QoS 2:

 This is the highest QoS level, used when neither loss nor duplication of messages is
acceptable.

 There is an increased overhead associated with this QoS level because each packet
contains an optional variable header with a packet identifier.

 Confirming the receipt of a PUBLISH message requires a two-step acknowledgement

process.

Dr. Syed Mustafa, HKBKCE.

204
 Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

QoS 2:

 The first step is done through the PUBLISH/PUBREC packet pair, and the second is
achieved with the PUBREL/PUBCOMP packet pair.

 This level provides a “guaranteed service” known as “exactly once” delivery, with no
consideration for the number of retries as long as the message is delivered once.

• QoS process is symmetric in regard to the roles of sender and receiver, but two separate
transactions exist.

• One transaction occurs between the publishing client and the MQTT server, and the other
Dr. Syed Mustafa, HKBKCE.
205
transaction happens between the MQTT server and the subscribing client.
 Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

Figure
provides an
overview of
the MQTT
QoS flows
for the three
different
levels.

Dr. Syed Mustafa, HKBKCE.

206
 Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

Dr. Syed Mustafa, HKBKCE.

207
Module – 3 Application Protocols for IoT
Message Queuing Telemetry Transport (MQTT):

Summary:

 MQTT is different from the “one-to-one” CoAP model in its “many-to-many”

subscription framework, which can make it a better option for some deployments.

 MQTT is TCP-based, and it ensures an ordered and lossless connection.

 It has a low overhead when optionally paired with UDP and flexible message format,
supports TLS for security, and provides for three levels of QoS.

 This makes MQTT a key application layer protocol for the successful adoption and
growth of the Internet of Things. Dr. Syed Mustafa, HKBKCE.
208