IoT Application Layer Protocols: CoAP and MQTT

The Internet of Things (IoT) industry is developing lightweight protocols like CoAP and MQTT to accommodate large numbers of constrained nodes and networks. CoAP is typically used over UDP, while MQTT is typically used over TCP. These lightweight protocols are being developed to address the issue of heavy web-based and data model protocols being too heavy for IoT applications.

---

CoAP

---

The Constrained Application Protocol (CoAP) is a framework developed by the IETF Constrained RESTful Environments (CoRE) working group to develop a generic framework for resource-oriented applications targeting constrained nodes and networks. 

 

The CoAP framework aims to provide simple and flexible ways to manipulate sensors and actuators for data or device management. The messaging model is designed to facilitate the exchange of messages over UDP between endpoints, including the secure transport protocol, Datagram Transport Layer Security (DTLS). 

 

The IETF CoRE working group is also studying alternate transport mechanisms, including TCP, secure TLS, and WebSocket. CoAP over Short Message Service (SMS) is also being considered for IoT device management. 

 

RFC 7252 provides more details on securing CoAP with DTLS, defining four security modes: NoSec, PreSharedKey, RawPublicKey, and Certificate. 

 

A CoAP message is composed of a short fixed-length header field, a variable-length but mandatory token field, options fields if necessary, and a payload field. The CoAP message format is simple and flexible, delivering low overhead and being easy to parse and process for constrained devices.

CoAP Message Format

CoAP, a communication protocol, can run over IPv4 or IPv6, but it is recommended to fit messages within a single IP packet and UDP payload to avoid fragmentation. 

 

For IPv6, the maximum CoAP message size could be up to 1152 bytes, including 1024 bytes for the payload. 

 

In IPv4, implementations should limit themselves to more conservative values and set the IPv4 Don't Fragment (DF) bit. CoAP doesn't rely on IP fragmentation but defines a pair of Block options for transferring multiple blocks of information from a resource representation in multiple request/response pairs. CoAP communications across an IoT infrastructure can take various paths, including between devices on the same or different constrained networks or between devices and generic Internet or cloud servers operating over IP. 

 

Proxy mechanisms are also defined, and RFC 7252 details a basic HTTP mapping for CoAP.

CoAP Communications in IoT Infrastructures

CoAP is a protocol that operates similarly to HTTP, but with a "thing" acting as both the client and server. It exchanges asynchronous messages, allowing a client to request an action via a method code on a server resource. A uniform resource identifier (URI) is localized on the server, and the server responds with a response code that may include a resource representation. 

 

CoAP has four types of messages: confirmable, non-confirmable, acknowledgement, and reset. The CoAP code, method and response codes, option numbers, and content format have been assigned by IANA as Constrained RESTful Environments (CoRE) parameters.

CoAP offers reliable transmission of messages when a CoAP header is marked as "confirmable." It also supports basic congestion control with a default time-out, simple stop and wait retransmission with exponential back-off mechanism, and detection of duplicate messages through a message ID. If a request or response is tagged as confirmable, the recipient must explicitly acknowledge or reject the message using the same message ID. If a recipient cannot process a non-confirmable message, a reset message is sent.

CoAP Reliable Transmission Example

The Figure above illustrates a utility operations center acting as the CoAP client, and a temperature sensor as the CoAP server. The communication between the client and server uses a CoAP message ID of 0x47, which ensures reliability and detects duplicate messages. In the Figure, the temperature sensor responds with an ACK message referencing the correct message ID of 0x47 and piggybacks a successful response to the GET request itself.

CoAP supports data requests sent to a group of devices by leveraging IP Multicast. Implementing IP Multicast requires the use of all-CoAP-node multicast addresses, such as 224.0.1.187 for IPv4 and FF0X::FD for IPv6. These multicast addresses are joined by CoAP nodes offering services to other endpoints while listening on the default CoAP port, 5683. Endpoints can find available CoAP services through multicast service discovery, such as deploying firmware upgrades for IoT devices like smart meters.

A CoAP server offering services and resources needs to be discovered by CoAP clients, either by learning a URI in a namespace or through the "All CoAP nodes" multicast address. The default port 5683 is used for non-secured CoAP (coap) and port 5684 for DTLS-secured CoAP (coaps). The CoAP server must be in listening state on these ports, unless a different port number is associated with the URI in a namespace.

CoAP specifications provide a description of the relationships between resources in RFC 6690, “Constrained RESTful Environments (CoRE) Link Format.” This standard defines the CoRE Link format carried as a payload with an assigned Internet media type. To improve response time and reduce bandwidth consumption, CoAP supports caching capabilities based on the response code. A wide range of CoAP implementations are available, some published with open source licenses and others part of vendor solutions.

---

Message Queuing Telemetry Transport (MQTT)

---

In the late 1990s, engineers from IBM and Arcom sought a reliable, lightweight, and cost-effective protocol for monitoring and controlling sensors in the oil and gas industries. Their research led to the development of the Message Queuing Telemetry Transport (MQTT) protocol, now standardized by the Organization for the Advancement of Structured Information Standards (OASIS). 

 

The protocol was designed to handle harsh environments, constrained nodes, unreliable WAN backhaul communications, and bandwidth constraints with variable latencies. The client/server and publish/subscribe framework was chosen based on the TCP/IP architecture, considering constrained nodes, unreliable WAN backhaul communications, and bandwidth constraints.

MQTT Publish/Subscribe Framework

 

An MQTT client can act as a publisher to send data or resource information to an MQTT server acting as an MQTT message broker. In an example, a temperature and relative humidity sensor publishes its Temp/RH data. The MQTT server accepts the network connection, application messages, and handles the subscription and unsubscription process. It pushes the application data to MQTT clients acting as subscribers.

The model where subscribers express a desire to receive information from publishers is well-known, such as Twitter. MQTT allows clients to subscribe to all or specific data from a publisher's information tree using a wildcard character. The presence of a message broker in MQTT decouples data transmission between publishers and subscribers, ensuring buffering and cached information in case of network failures. This allows publishers and subscribers to be online simultaneously.

MQTT control packets run over a TCP transport using port 1883, ensuring an ordered, lossless stream of bytes between the MQTT client and the MQTT server. MQTT can be secured using TLS on port 8883 or WebSocket (defined in RFC 6455). Each control packet consists of a 2-byte fixed header with optional variable header fields and optional payload, with a payload up to 256 MB.

MQTT Message Format

MQTT is a message format with a smaller header of 2 bytes compared to CoAP. It includes a Message Type field, which identifies the type of MQTT packet within a message. MQTT version 3.1.1 specifies fourteen different types of control packets, each with a unique value coded into the Message Type field. The QoS header field allows for the selection of three different QoS levels. The Retain flag, only found in PUBLISH messages, notifies the server to hold onto the message data, allowing new subscribers to receive the last known value without waiting for the next update from the publisher. The last mandatory field is Remaining Length, specifying the number of bytes in the MQTT packet.

MQTT sessions between clients and servers consist of four phases: session establishment, authentication, data exchange, and session termination. Each client connecting to a server has a unique client ID, allowing the identification of the MQTT session between both parties. Subscriptions to resources generate SUBSCRIBE/SUBACK control packets, while unsubscription is performed through the exchange of UNSUBSCRIBE/UNSUBACK control packets. Graceful termination of a connection is done through a DISCONNECT control packet, which also offers the capability for a client to reconnect by re-sending its client ID to resume operations.

A message broker uses a topic string or topic name to filter messages for its subscribers. When subscribed to a resource, the subscriber indicates one or more topic levels to structure the topic name. The forward slash (/) in an MQTT topic name separates each level within the topic tree, providing a hierarchical structure to the topic names.

MQTT QoS Flows

The Quality of Service (QoS) process in the Internet of Things (IoT) involves symmetric roles for sender and receiver, but two separate transactions occur. One occurs between the publishing client and the MQTT server, while the other occurs between the MQTT server and the subscribing client. MQTT implementations are available as open source licenses or integrated into vendors' solutions like Facebook Messenger.

Both MQTT and CoAP have their strengths and weaknesses in an IoT network. MQTT is different from the "one-to-one" CoAP model in its "many-to-many" subscription framework, making it a better option for some deployments. It is TCP-based, ensures an ordered and lossless connection, has low overhead when paired with UDP, supports TLS for security, and provides three levels of QoS. This makes MQTT a key application layer protocol for the successful adoption and growth of the Internet of Things.

---

Happy Exploring!