Firmware updates: the SFT protocol using HTTP for downloading files

If you need to use HTTP download with our Secure File Transfer protocol, follow these guidelines.

By default, the entire firmware update process flow is carried out over MQTT, but files larger than 8MB can only be downloaded over HTTP/HTTPS by the application. The way of downloading is eligible by device.

Updates are managed through two MQTT control topics:

  • xi/ctrl/v1/<deviceId>/cln Xively sends messages to the device on this topic (devices should always subscribe to this topic if they wish to use firmware update capabilities)
  • xi/ctrl/v1/<deviceId>/svc the device sends messages to Xively on this topic
  • Device must publish to the /svc topic using QOS0 or the broker will not respond.

There are 3 types of messages used in firmware and file deployment over HTTP:

Message type
Message number
Direction
Description

FILE_INFO

0

Device -> Broker

Used by the device to report what files it has, and at what version.

FILE_UPDATE_AVAILABLE

1

Broker -> Device

Used by the broker to tell a device what file updates are available (if any).

FILE_STATUS

4

Device -> Broker

Used by the device to report on the success or failure of a file's delivery. 'Success' is based on whether the downloaded file matches the fingerprint checksum that the broker sent along with the file in the FILE_UPDATE_AVAILABLE message.

All messages to and from the broker are encoded as RFC 7049 Concise Binary Object Representation (CBOR) packets. See the RFC for specifications and cbor.io for example codecs.

The flow

  1. Devices can indicate that they are capable of downloading files directly through HTTP/HTTPS by sending the L flag with value true in the FILE_INFO message.

Limited number of files in a given fileset

Each fileset can consist of a maximum 16 pcs of files = 16 pcs of download links in this case.

  1. In case of getting 'L' flag, the broker will send the download link in FILE_UPDATE_AVAILABLE messages.

  2. Per file: HTTP/HTTPS downloading is managed by the embedded application. Application is responsible to perform the checksum validation.

  3. Per file: As a difference to the normal MQTT flow the DOWNLOADING state also must be reported by the device in this case.

You need to host the files to be downloaded.

The Xively client can handle file updates in the background

The Xively embedded client (written in C), is the only embedded client on the market that is capable of multitasking while remaining single-threaded.

This means that (unlike all other lightweight embedded clients that use MQTT), it is capable of downloading file updates in the background without interfering with the devices standard operations.

Our client also supports HTTP downloading option.

The 'FILE_INFO' message (0)

Devices send the FILE_INFO message periodically to check for available file updates and after performing an update.

These messages consist of:

  • A message type field equal to FILE_INFO
  • A FILE_INFO message version number
  • An array of File name and Revision
  • The 'L' value indicates that you are trying to get a link to the HTTP/HTTPS download
 [CBOR_TYPE_MAP] Definite, size: 3
    [CBOR_TYPE_STRING] Definite, "msgtype"
    [CBOR_TYPE_UINT] Value 0
    [CBOR_TYPE_STRING] Definite, "msgver"
    [CBOR_TYPE_UINT] Value 1
    [CBOR_TYPE_STRING] Definite, "L"
    [CBOR_TYPE_CTRL] Value CBOR_CTRL_TRUE
    [CBOR_TYPE_STRING] Definite, "list"
    [CBOR_TYPE_ARRAY] Indefinite, size:  2
        [CBOR_TYPE_MAP] Definite, size: 2
            [CBOR_TYPE_STRING] Definite, "N"
            [CBOR_TYPE_STRING] Definite, "OS"
            [CBOR_TYPE_STRING] Definite, "R"
            [CBOR_TYPE_STRING] Definite, "1_0_5"
        [CBOR_TYPE_MAP] Definite, size: 2
            [CBOR_TYPE_STRING] Definite, "N"
            [CBOR_TYPE_STRING] Definite, "BSP"
            [CBOR_TYPE_STRING] Definite, "R"
            [CBOR_TYPE_STRING] Definite, "Latest"
{
    "msgtype": 0,
    "msgver": 1,
    "L": true,
    "list": [
        {"N": "OS", "R": "1_0_5"},   
        {"N": "BSP", "R": "Latest"}
    ]
}

The 'FILE_UPDATE_AVAILABLE' message (1)

The FILE_UPDATE_AVAILABLE message is sent to devices in response to a FILE_INFO message when updates are available for the device to download. The device can start updating immediately, wait until it is idle, or wait for user consent.

This message consists of:

  • A message type field equal to FILE_UPDATE_AVAILABLE
  • A FILE_UPDATE_AVAILABLE message version
  • An array of:
    • File name
    • Revision
    • ImageSize
    • Fingerprint
  • DownloadLink (optional, only present if L was true in FILE_INFO)
  • DownloadMQTT (optional, only present if L was true in FILE_INFO, indicates whether it is still possible to download the file over MQTT or not, decided by Xively broker based on file size)
[CBOR_TYPE_MAP] Definite, size: 3
    [CBOR_TYPE_STRING] Definite, msgtype
    [CBOR_TYPE_UINT] Value: 1
    [CBOR_TYPE_STRING] Definite, msgver
    [CBOR_TYPE_UINT] Value: 1
    [CBOR_TYPE_STRING] Definite, list
    [CBOR_TYPE_ARRAY] Indefinite, size:  2
        [CBOR_TYPE_MAP] Definite, size: 4
            [CBOR_TYPE_STRING] Definite, N
            [CBOR_TYPE_STRING] Definite, OS
            [CBOR_TYPE_STRING] Definite, R
            [CBOR_TYPE_STRING] Definite, 1_0_5
            [CBOR_TYPE_STRING] Definite, S
            [CBOR_TYPE_UINT] Width: 4B, Value: 12345
            [CBOR_TYPE_STRING] Definite, F
            [CBOR_TYPE_BYTESTRING] Definite, 561516521652165
            [CBOR_TYPE_STRING] Definite, L
            [CBOR_TYPE_STRING] Definite, https://example.com/firmware/OS_1_0_5.bin
            [CBOR_TYPE_STRING] Definite, M
            [CBOR_TYPE_CTRL] Value CBOR_CTRL_FALSE
        [CBOR_TYPE_MAP] Definite, size: 4
            [CBOR_TYPE_STRING] Definite, N
            [CBOR_TYPE_STRING] Definite, Credentials
            [CBOR_TYPE_STRING] Definite, R
            [CBOR_TYPE_STRING] Definite, 1_0_0
            [CBOR_TYPE_STRING] Definite, S
            [CBOR_TYPE_UINT] Width: 4B, Value: 230
            [CBOR_TYPE_STRING] Definite, F
            [CBOR_TYPE_BYTESTRING] Definite, 11001001010
            [CBOR_TYPE_STRING] Definite, L
            [CBOR_TYPE_STRING] Definite, https://example.com/firmware/Credentials_1_0_0.bin
            [CBOR_TYPE_STRING] Definite, M
            [CBOR_TYPE_CTRL] Value CBOR_CTRL_TRUE
{
    "msgtype": 1,
    "msgver": 1,
    "list": [
        {"N":"OS","R": "1_0_5", "S":12345, "F": "561516521652165","L":"https://example.com/firmware/OS_1_0_5.bin","M":false},
        {"N":"Credentials","R": 1_0_0", "S":230, "F": "11001001010","L":"https://example.com/firmware/Credentials_1_0_0.bin","M":true}
    ]
}

The 'FILE_STATUS message (4)

The FILE_STATUS message reports the status of a single file. Status codes other than zero (i.e. success) are application-dependent. For example: -3: File lost -14: File Corrupted

These messages in relation to a given file are expected to arrive in the following order: downloaded, processing, done, but any of these may be skipped.

If status messages arrive in a different order, the report status will be undefined. If the FILE_UPDATE_AVAILABLE message contained multiple files, the overall progress of the update process is considered to be that of the 'smallest' of all files (available < downloading < downloaded < processing < done, where available and downloading are reported by Xively itself).

A message type field equal to FILE_STATUS
Message version
File Name
Revision
File Status Phase: uint8
0-1 - Reserved, set by Xively itself, ignored when received from clients (Unreported, Reported, Downloading)
2 - Downloading (this value must only be sent if the file is not downloaded over MQTT, it should only be sent once for each file download attempt
3 - Downloaded
4 - Processing
5 - Finished (the client will not work on the file anymore, even after an error)
File Status code: int32
If it is greater than or equal to 0, the result returned is a Success
If it is less than 0, the result returned is an Error and if the deployment returns a message less than 0, then it will appear under 'Finished with error'

The Default integer is 0.
You can view the status of your Production Deployments under the Xively web interface. For more information, see Monitoring a Production Deployment. Information relating to Device specific error codes will appear in the Device logs, see Viewing Device Firmware.

In CBOR, it looks like:

 [CBOR_TYPE_MAP] Definite, size: 6
    [CBOR_TYPE_STRING] Definite, msgtype
    [CBOR_TYPE_UINT]  Value: 4
    [CBOR_TYPE_STRING] Definite, msgver
    [CBOR_TYPE_UINT]  Value: 1
    [CBOR_TYPE_STRING] Definite, N
    [CBOR_TYPE_STRING] Definite, OS
    [CBOR_TYPE_STRING] Definite, R
    [CBOR_TYPE_STRING] Definite, 1_0_5
    [CBOR_TYPE_STRING] Definite, P
    [CBOR_TYPE_UINT] Width: 1B, Value: 5
    [CBOR_TYPE_STRING] Definite, S
    [CBOR_TYPE_UINT] Width: 1B, Value: 0
{
"msgtype": 4,
"msgver": 1,
"N": "OS",
"R": "1_0_5",
"P":5
"S":0
}