Firmware updates (device side)

How to set up your devices to be notified of - and updated with - the latest firmware and file updates

From the device side, firmware and file updates are built to be very easy and flexible to implement, with Xively managing your files' version histories and deployments to individual devices, and rolling that information up to admins in the management app.

See the guide on firmware updates from the admin side for more on this process.

What's the general process?

  1. When a device wakes up, it reports on the latest versions of files it has (in a FILE_INFO message)
  2. If there are newer versions available for any of the files the device has reported, Xively will report back the latest versions: their names, revision numbers, size and fingerprints or checksums (in a FILE_UPDATE_AVAILABLE message)
  3. The device can cycle through the available files to update and request them in chunks (with a FILE_GET_CHUNK message). The device can also refuse the update (with a FILE_STATUS message).
  4. The broker will send back chunked payloads of the files with the requested offset and length (in a FILE_CHUNK message)
  5. (Optional) The device can report back a successful or failed download on a per-file basis when it receives the full file and validates its checksum locally (with a FILE_STATUS message)
  6. When the device has received all of the updates it wants at that time and validated those files, it can update Xively with its latest file statuses (with another FILE_INFO message)

Rollouts of new firmware

When an admin deploys new firmware as a test deployment, devices will receive an updated list of files the next time they check with the broker using a FILE_INFO packet.\n\nWhen an admin deploys new firmware as a fleet-wide deployment, devices will receive an updated list of files some time over the next 24 hours the next time they check with the broker using a FILE_INFO packet. Some devices will not receive the updates list for up to 24 hours after the deployment starts.

Sequence diagram

The below diagram follows the firmware/file deployment process from the moment of a device powering on and asking the broker if there are any files that it should update, through to the device's confirmation of updated files and installation/reboot.

Message types overview

There are 5 types of messages used in firmware and file deployment:

Message type
Message number
Direction
Description

FILE_INFO

0

Device -> Broker

Used by the device to report what files it has, and at what version.

FILE_UPDATE_AVAILABLE

1

Broker -> Device

Used by the broker to describe to a device what file updates are available (if any).

FILE_GET_CHUNK

2

Device -> Broker

Used by the device to request chunks of files it would like to update.

FILE_CHUNK

3

Broker -> Device

Used by the broker to deliver chunks of files to the device.

FILE_STATUS

4

Device -> Broker

Used by the device to report on the success or failure of a file's delivery. 'Success' is based on whether the downloaded file matches the fingerprint checksum that the broker sent along with the file in the FILE_UPDATE_AVAILABLE message.

How it's done over MQTT

The entire firmware update process is carried out over MQTT, meaning that the device never has to switch contexts or interrupt its normal operation while downloading files.

All messages to and from the broker are encoded as RFC 7049 Concise Binary Object Representation (CBOR) packets. See the RFC for specifications and cbor.io for example codecs.

The Xively client can handle file updates in the background

The Xively embedded client (written in C), is the only embedded client on the market that is capable of multitasking while remaining single-threaded.

This means that (unlike all other lightweight embedded clients that use MQTT), it is capable of downloading file updates in the background while maintaining its current subscriptions, continuing to publish messages, and allowing user interaction from buttons and interfaces on the device.

Updates are managed through two MQTT control topics:

  • xi/ctrl/v1/<deviceId>/cln Xively sends messages to the device on this topic (devices should always subscribe to this topic if they wish to use firmware update capabilities)
  • xi/ctrl/v1/<deviceId>/svc the device sends messages to Xively on this topic
  • Device must publish to the /svc topic using QOS0 or the broker will not respond.

The 'FILE_INFO' message (0)

Devices send this message periodically to check for available file updates and after performing an update.

These messages consist of:

  • A message type field equal to FILE_INFO
  • A FILE_INFO message version number
  • An array of File name and Revision
[CBOR_TYPE_MAP] Definite, size: 3
    [CBOR_TYPE_STRING] Definite, "msgtype"
    [CBOR_TYPE_UINT] Value 0
    [CBOR_TYPE_STRING] Definite, "msgver"
    [CBOR_TYPE_UINT] Value 1
    [CBOR_TYPE_STRING] Definite, "list"
    [CBOR_TYPE_ARRAY] Indefinite, size:  2
        [CBOR_TYPE_MAP] Definite, size: 2
            [CBOR_TYPE_STRING] Definite, "N"
            [CBOR_TYPE_STRING] Definite, "OS"
            [CBOR_TYPE_STRING] Definite, "R"
            [CBOR_TYPE_STRING] Definite, "1_0_5"
        [CBOR_TYPE_MAP] Definite, size: 2
            [CBOR_TYPE_STRING] Definite, "N"
            [CBOR_TYPE_STRING] Definite, "BSP"
            [CBOR_TYPE_STRING] Definite, "R"
            [CBOR_TYPE_STRING] Definite, "Latest"

The 'FILE_UPDATE_AVAILABLE' message (1)

This message is sent to devices in response to a FILE_INFO message when updates are available for the device to download. The device can start updating immediately, wait until idle, or wait for user consent.

This message consists of:

  • A message type field equal to FILE_UPDATE_AVAILABLE
  • A FILE_UPDATE_AVAILABLE message version
  • An array of:
    • File name
    • Revision
    • ImageSize
    • Fingerprint
[CBOR_TYPE_MAP] Definite, size: 3
    [CBOR_TYPE_STRING] Definite, msgtype
    [CBOR_TYPE_UINT] Value: 1
    [CBOR_TYPE_STRING] Definite, msgver
    [CBOR_TYPE_UINT] Value: 1
    [CBOR_TYPE_STRING] Definite, list
    [CBOR_TYPE_ARRAY] Indefinite, size:  2
        [CBOR_TYPE_MAP] Definite, size: 4
            [CBOR_TYPE_STRING] Definite, N
            [CBOR_TYPE_STRING] Definite, OS
            [CBOR_TYPE_STRING] Definite, R
            [CBOR_TYPE_STRING] Definite, 1_0_5
            [CBOR_TYPE_STRING] Definite, S
            [CBOR_TYPE_UINT] Width: 4B, Value: 12345
            [CBOR_TYPE_STRING] Definite, F
            [CBOR_TYPE_STRING] Definite, 561516521652165
        [CBOR_TYPE_MAP] Definite, size: 4
            [CBOR_TYPE_STRING] Definite, N
            [CBOR_TYPE_STRING] Definite, Credentials
            [CBOR_TYPE_STRING] Definite, R
            [CBOR_TYPE_STRING] Definite, 1_0_0
            [CBOR_TYPE_STRING] Definite, S
            [CBOR_TYPE_UINT] Width: 4B, Value: 230
            [CBOR_TYPE_STRING] Definite, F
            [CBOR_TYPE_STRING] Definite, 11001001010

The 'FILE_GET_CHUNK' message (2)

Devices send this message during the update process to fetch a chunk of a file (i.e. the first message of this kind after a FILE_UPDATE_AVAILABLE indicates that the device starts the file update process). Length must be smaller than or equal to 65536.

These messages consist of:

  • A message type field equal to FILE_GET_CHUNK
  • FILE_GET_CHUNK message version
  • File name
  • Revision
  • Offset: uint32
  • Length: uint32
[CBOR_TYPE_MAP] Definite, size: 6
     [CBOR_TYPE_STRING] Definite, msgtype
    [CBOR_TYPE_UINT] Value: 2
    [CBOR_TYPE_STRING] Definite, msgver
    [CBOR_TYPE_UINT] Width: 1B, Value: 1
    [CBOR_TYPE_STRING] Definite, N
    [CBOR_TYPE_STRING] Definite, OS
    [CBOR_TYPE_STRING] Definite, R
    [CBOR_TYPE_STRING] Definite, 1_0_5
    [CBOR_TYPE_STRING] Definite, O
    [CBOR_TYPE_UINT] Width: 4B, Value: 12345
    [CBOR_TYPE_STRING] Definite, L
    [CBOR_TYPE_UINT] Width: 4B, Value: 4096

The 'FILE_CHUNK' message (3)

This message is sent in response to a FILE_GET_CHUNK message with the requested chunk of the given file.

These messages consist of:

  • A message type field equal to FILE_CHUNK
  • FILE_CHUNK message version
  • File Name
  • Revision
  • Offset: uint32
  • Length: uint32
  • Status: uint8Chunk: byte array

The status field indicates success or error as follows:
Status = 0: File chunk request is correct.
Status = 1: This block contains the end of file and so it's shorter than the request length.
Status = 2: File offset is greater then file length.
Status = 3: Requested length is greater than the maximum allowed length of 65536 bytes.
Status = 4: File unavailable.

 [CBOR_TYPE_MAP] Definite, size: 8
    [CBOR_TYPE_STRING] Definite, msgtype
    [CBOR_TYPE_UINT]  Value: 3
    [CBOR_TYPE_STRING] Definite, magver
    [CBOR_TYPE_UINT]  Value: 1
    [CBOR_TYPE_STRING] Definite, N
    [CBOR_TYPE_STRING] Definite, OS
    [CBOR_TYPE_STRING] Definite, R
    [CBOR_TYPE_STRING] Definite, 1_0_5
    [CBOR_TYPE_STRING] Definite, O
    [CBOR_TYPE_UINT] Width: 4B, Value: 12345
    [CBOR_TYPE_STRING] Definite, L
    [CBOR_TYPE_UINT] Width: 4B, Value: 4096
    [CBOR_TYPE_STRING] Definite, S
    [CBOR_TYPE_UINT] Width: 1B, Value: 0
    [CBOR_TYPE_STRING] Definite, C
    [CBOR_TYPE_BYTESTRING] Definite, size: 4096
        [CBOR_TYPE_UINT] Width: 1B, Value: 167
        [CBOR_TYPE_UINT] Width: 1B, Value: 241
                  ...
        [CBOR_TYPE_UINT] Width: 1B, Value: 217
        [CBOR_TYPE_UINT] Width: 1B, Value: 42
        [CBOR_TYPE_UINT] Width: 1B, Value: 130

The 'FILE_STATUS message (4)

These messages consist of:

  • A message type field equal to FILE_STATUS
  • FILE_STATUS message version
  • File Name
  • Revision
  • File Status Message (String)
  • File Status code: int32
    • 0 - Download success
    • 1 - File removed
    • 2 - File updated
    • 100 - Fingerprint mismatch
    • 101 - Unrecognized file name
    • 102 - Cancelled
    • 104 - Cannot flash105 - File too large
 [CBOR_TYPE_MAP] Definite, size: 4
    [CBOR_TYPE_STRING] Definite, msgtype
    [CBOR_TYPE_UINT]  Value: 4
    [CBOR_TYPE_STRING] Definite, magver
    [CBOR_TYPE_UINT]  Value: 1
    [CBOR_TYPE_STRING] Definite, N
    [CBOR_TYPE_STRING] Definite, OS
    [CBOR_TYPE_STRING] Definite, R
    [CBOR_TYPE_STRING] Definite, 1_0_5
    [CBOR_TYPE_STRING] Definite, S
    [CBOR_TYPE_UINT] Width: 1B, Value: 0

Firmware updates (device side)

How to set up your devices to be notified of - and updated with - the latest firmware and file updates