Variable Header


Various message types require additional information as part of the header section of the message, before the application-defined Payload. The fields that are present in each type of message are defined in the specification for that message type. The complete collection of fields and their data types is specified here.

In a message where several Variable Header fields appear, they must appear strictly in the order specified in this section.

Note that the variable-length Remaining Length field is not part of the Variable Header, and so its bytes do not contribute to the byte count of the Remaining Length value, which only considers the Variable Header and Payload portions of the message. See the Fixed Header section for more information.
 

Protocol Name

Present in: CONNECT

UTF-encoded string to identify the name of the protocol that is being used.
This is defined to be "MQIpdp", capitalised as shown. The letters stand for Message Queuing Integrator Pervasive Device Protocol, for what it's worth.

See the UTF section for more information.
 
 

Protocol Version

Present in: CONNECT

The Version field indicates the current revision level of the protocol being used by the client. The Version field is an 8 bit unsigned value representing the Protocol Version. For this version of the protocol, the Version field will contain a value of 2 (0x02).
 
 
bit
7
6
5
4
3
2
1
0
 
Protocol Version
 
0
0
0
0
0
0
1
0

 

Connect Flags

Present in: CONNECT

The Connect Flags allow various options to be selected when a client connects to the broker.
 

Topic Name COMPression (TNCOMP) Request

Present in: CONNECT

The TNCOMP Request flag is bit 0 of the Connect Flags byte.

The TNCOMP Request flag is used to indicate that the client wishes to make use of the Topic Name compression  algorithm (described below). A 1 indicates that the client may wish to compress Topic Names, a 0 indicates that the client will never compress a Topic Name.

In this level of the Protocol, Topic Name compression is not available, and so the value of this flag is always 0.
 
bit
7
6
5
4
3
2
1
0
 
reserved
reserved
Will RETAIN
Will QoS
Will Flag
Clean Start
TNCOMP
Request
 
x
x
x
x
x
x
 x
0

The upper  2 bits of this byte are currently unused in this version of the protocol. They are, however reserved for future use.
 

Clean Start flag

Present in: CONNECT

The Clean Start flag is bit 1 of the Connect Flags byte.

The Clean Start flag provides a way for the client to go back to a known, "clean" state with the broker. If the flag is set, the broker will discard any outstanding messages for that client, delete all subscriptions for that client, and reset the Message ID to 1. The client will then be able to proceed without risk the of any data from previous connections interfering with the current connection.
 
 
bit
7
6
5
4
3
2
1
0
 
reserved
reserved
Will RETAIN
Will QoS
Will Flag
Clean Start
TNCOMP
Request
 
x
x
x
x
x
x
 
x

The upper  2 bits of this byte are currently unused in this version of the protocol. They are, however reserved for future use.
 

"Will" flag

Present in: CONNECT

The "Will" flag is bit 2 of the Connect Flags byte.

The "Will" message is a mechanism by which a client can define a message to be published on its behalf by the broker, in the event of the client's
"death". We define "death" in this context to mean that either an I/O error is encountered by the broker during communication with the client, or
the client fails to communicate within the Keep Alive Timer schedule (see below). The receipt of a DISCONNECT packet from the client does
NOT trigger the sending of the Will message.

If the Will flag is set, then the Will QoS and Will RETAIN fields in the Connect Flags byte must be provided, as must the Will Topic and Will Message fields in the Payload.
 
 
bit
7
6
5
4
3
2
1
0
 
reserved
reserved
Will RETAIN
Will QoS
Will Flag
Clean Start
TNCOMP
Request
 
x
x
x
x
x
 
 x
x

The upper  2 bits of this byte are currently unused in this version of the protocol. They are, however reserved for future use.
 

"Will" QoS

Present in: CONNECT

The "Will" QoS field is bits 4 and 3 of the Connect Flags byte.

The Will QoS field is used by a connecting client to specify the Quality of Service at which the Will Message (part of the Payload of the CONNECT message) will be sent in the event of the untimely death (involuntary disconnection) of this client.

If the Will Flag is set, then this field is mandatory, otherwise its value is disregarded.

Will QoS can take the value 0 (0x00), 1 (0x01), or 2 (0x02), just like an ordinary QoS on a PUBLISH.
 
 
bit
7
6
5
4
3
2
1
0
 
reserved
reserved
Will RETAIN
Will QoS
Will Flag
Clean Start
TNCOMP
Request
 
x
x
x
   
1
 x
x

The upper  2 bits of this byte are currently unused in this version of the protocol. They are, however reserved for future use.
 

"Will" RETAIN flag

Present in: CONNECT

The "Will" RETAIN flag is bits5 of the Connect Flags byte.

The Will RETAIN flag is used to indicate if the Will Message, published by the broker on behalf of the client if it becomes unexpectedly disconnected, is to be RETAINed by the broker or not. See discussion of RETAIN in the PUBLISH section.

If the Will Flag is set, then this field is mandatory, otherwise its value is disregarded.
 
 
bit
7
6
5
4
3
2
1
0
 
reserved
reserved
Will RETAIN
Will QoS
Will Flag
Clean Start
TNCOMP
Request
 
x
x
 
x
x
1
 x
x

The upper  2 bits of this byte are currently unused in this version of the protocol. They are, however reserved for future use.
 
 

Keep Alive Timer

Present in: CONNECT

The Keep Alive timer is measured in seconds, and is the means by which a client tells the broker the maximum time interval between receiving messages from this client. This gives the broker a way of detecting that the network connection to a client has dropped, without having to wait for the long TCP/IP timeout. It is the responsibility of the client to ensure that it sends some kind of message within each Keep Alive time period. If there is not going to be a data-related message within the time period, then the client should send a PINGREQ message, which will be acknowledged by the broker with a PINGRESP.

If the broker does not hear from this client within 1.5 times the Keep Alive time period (i.e. the client is allowed half a time period's "grace"), it performs the disconnection processing that it would have done if the client had sent a DISCONNECT message. Note that this does not impact any of the client's subscriptions. See the DISCONNECT section for more details.

The Keep Alive timer is a 16-bit value representing the number of seconds for the timeout. This allows timeouts up to around 18 hours.  The appropriate timeout value is application specific, but it is anticipated  that timeouts would be of the order of a few minutes.
A Keep Alive time of 0 indicates that there will be no Keep Alive timeout processing, i.e. the broker will not disconnect a client due to elapsed time.
The ordering of the 2 bytes of the Keep Alive Timer is MSB, then LSB (big-endian).
 
 
bit
7
6
5
4
3
2
1
0
 
Keep Alive MSB
 
Keep Alive LSB

 

Topic Name COMPression (TNCOMP) Response

Present in: CONNACK

The TNCOMP Response flag is used to indicate that the broker has granted the client's request to make use of the Topic Name compression  algorithm (described below). A 1 indicates that the client is permitted to compress Topic Names, a 0 indicates that the client must not compress any Topic Name.

In this level of the Protocol, Topic Name compression is not available, and so the value of this flag is always 0.
 
bit
7
6
5
4
3
2
1
0
 
reserved
reserved
reserved
reserved
reserved
reserved
reserved
TNCOMP
Response
 
x
x
x
x
x
x
x
0

The upper  7 bits of this byte are currently unused in this version of the protocol. They are, however reserved for future use.
 

Return Code

Present in: CONNACK

This field defines a one byte unsigned return code. The meaning of the various values are specific to the message type.
A return code of 0 can usually be expected to mean "success".

See the individual message specifications for more information.
 
 
bit
7
6
5
4
3
2
1
0
 
Return Code

 

Topic Name

Present in: PUBLISH

For a PUBLISH message, the Topic Name is the key indicates the information channel to which the data in the Payload is being published. It is the key which subscribers use to identify the information channels on which they wish to receive published information.

The Topic Name has an upper length limit of 32,767 characters. This should not be cause any problems in practice.

The Topic Name can take one of two forms, according to the setting of the TNCOMP flag in the CONNECT message, and subsequently confirmed in the CONNACK message. See the relevant sections for full details.
 

CASE 1

If TNCOMP (Topic Name COMPression) is not set, then no compression of Topic Names will take place during this session, and the format of the Topic Name is simply a UTF-encoded string.

See the UTF section for more information.
 

CASE 2

This has not yet been thoroughly thought through... I suggest that in Version 1 and 2 of the protocol, we define TNCOMP to be 0 for all PUBLISH messages.

If TNCOMP (Topic Name COMPression) is set, then the Topic Name may be compressed to potentially reduce the size of the message. Even though compression is permitted in the session, the Topic Name is not necessarily compressed. The topmost bit of the first byte of the Topic Name string is used to indicate if compression has been used. A 0 in this position indicates no compression, and the Topic Name is a simple UTF representation. A 1 in this position indicates that the string has been compressed in the following manner.

The content of the string (after the length bytes) is first of all XOR-d with the string that was previously sent. Care must be taken in a multi-process or multi-threaded environment that buffer contents are in sync at the client and broker, possibly by implementing this compression at the protocol library level, rather than in the application.

The resulting string is the run-length encoded... need to think some more about this!
 
 

Message Identifier

Possibly present in:PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK.

The Message Identifier field is only present in messages where the QoS bits in the Fixed Header indicate QoS 1 or QoS 2. See the QoS section for more information.

The Message Identifer is a 16 bit unsigned integer, which will typically increase, but is not required to increase by exactly one from one message to the next. This assumes that there will never be more than 65,535 messages "in flight" between one particular client-broker pair at any time.

The ordering of the 2 bytes of the Message Identifier is MSB, then LSB (big-endian).

Message ID 0 is reserved as an invalid Message ID, and must not be used.
 
bit
7
6
5
4
3
2
1
0
 
Message Identifier MSB
 
Message Identifier LSB


Discussion

ASC on endianism
by popular demand from the Java programmers, the Message ID and the Keep Alive timer have been changed back to network ordering, big-endian notation: MSB, LSB.
AN on Message Identifier
Although specified as a value unique to any other message pending in the system, it should suffice that this number be defined as a binary incrementing number starting at a value of 0x0001 and incrementing to a value of 0xFFFF before rolling over to a value of 0x0001 again. This would also consider a value of 0x0000 to be an invalid sequence number. This assumes that there would never be a time where 65,535 messages would be outstanding to be acknowledged. ( I would also like to know from the Argo perspective whether there is value in this number being a 32 bit value. In the current Ambrosia implementation this value is a 32 bit number respresenting the number of milli-seconds that the client side has been running from power up, but I don’t know that it was ever actually used for anything.)
ASC comment
Currently unresolved whether we should not use 0x0000.
ASC comment
Version 2 of the protocol reserves MID 0 as an invalid Message ID.

 
ASC on Message Identifier
Should the application SET the Message Identifier?
Should the application SEE the Message Identifier?
Do we even NEED a Message Identifier? This boils down to concurrent writes, versus sequential blocking: i.e. do you wait for an :PUBACK for a packet before sending the next one, or is it desirable to transmit a sequence of messages and gather PUBACKs afterwards?
Comment
No, the protocol library should set the Message Identifier, but in instances where there is a discernable boundary between application and protocol implementation, then the Identifier should be passed back to the application as a handle.
Yes, can be passed back as the message handle.
Yes, since there can be outstanding acknowledgements after multiple PUBLISH messages.

 
ASC on TTE (Time To Expire) flag
For broker to client messages, could the broker do the testing for timeout and abandon delivery attempts if the TTE has expired? Compare and contrast TTE-based delivery with At Most Once ("Fire-and-Forget") delivery: is it actually used to retry an operation until a certain time, or is it really an implementation of the At Most Once quality of service? What about retry attempts? Does the client (or broker) attempt retries while the message is still in the TTE window, or is it literally try it once and if it fails, abandon it? For the expected TTE values, does it make sense to have epoch at 1/1/80, or could we move it forward and buy more time? Also, would there be any merit in reducing the size of the field, and hence have it wrap around earlier?
AN comment
These issues are not appropriate to the protocol layer. If anything, it is an application level thing, and so is not the concern of the protocol, and thus will not appear in the Fixed or Variable Headers.

 
ASC on Topic Name
Note that we use the first bit of the first byte of the string representation to indicate whether or not the string is compressed. For UTF encoding of a string of less than 32,767 characters, the topmost bit of the first byte of the string is 0, and so this is used to indicate no compression of Topic Name. A 1 in this position indicates that the string is compressed with the XOR and RLE method desribed above.

 
ASC on Message Identifier
The Message Identifier is after the Topic Name, so that on receiving a PUBLISH message, the recipient can parse the Topic Name regardless of whether the QoS bits are set to indicate the presence of the Message Identifier field.
BACK to index 
Last Modified: 22-Oct-99