Data Formats in our Cloud

What is CBOR, ABCL and JSON payload?

Omar Cruz avatar
Written by Omar Cruz
Updated over a week ago

Data formats

Devices can use different formats to send and receive data from and to AllThingsTalk Cloud. The Cloud supports both verbose formats — like JSON, and formats more suitable to constrained devices and networks — like CBOR. If needed, custom formats can be defined using ABCL.

Supported formats

Our Arduino-based SDKs support JSON, CBOR and custom binary data format.

JSON

If your project involves an IP-enabled device and you plan to use HTTP or MQTT protocols, you typically want to use the JSON formatted data. JSON is a lightweight data-interchange format which is easy for humans to read and write and easy for machines to parse. It’s widely adopted on the web.

CBOR

If your device operates on a constrained network, for example LoRa, Sigfox or NB-IOT, parsing and constructing JSON becomes difficult. This is when it’s better to use a lightweight binary format such as CBOR.

CBOR is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation.

→ See RFC 7049 for the CBOR specification, and cbor.io for more background information.

Custom binary

In case you want to connect an off-the-shelf device or define your own binary payload format, you might consider using AllThingsTalk Binary Conversion Language (ABCL). ABCL is a data description language that allows you to support any custom binary data format.

States vs com mands

AllThingsTalk Cloud allows clients to send states — uplink, and receive commands — downlink.

  • State defines a particular condition that an asset (or more generally a device) is in at a specific time. Devices (or other clients) usually send their sensor state to the cloud for further processing.

  • Command is an instruction or signal that causes a device to perform an actuating function. Devices receive commands being dispatched over the cloud.

Sending states

For uplink scenarios, we accept asset states and device states.

Asset states
Asset state defines a condition that an asset is in at a specific time. For example, a device with temperature sensor can measure a condition of 33 degrees Celsius at 12am.

The Cloud expects asset states to arrive through HTTP or MQTT at:

  • asset/<AssetID>/state

  • device/<DeviceID>/asset/<AssetName>/state

JSON format
When your device are not restricted by power or bandwidth, the easiest way to send data to AllThingsTalk Cloud is by using the JSON format. For example:

{
  "value" : 33,
  "at": "2018-03-28T12:00Z+02:00"
}
  • value — required; a measured state value. Type could be any JSON value (string, number, object, array, true, false or null) that is described in the related asset profile.

  • at — measurement timestamp as ISO 8601 formatted string. If left out, the Cloud will assign time the message has been received.

Here are several examples of asset state messages:

Integer

{ "value": 10 }

String

{ "value": "pancakes" }

Object

{ "value": {"latitude": 3.0735, "longitude": 51.45678} }

String with timestamp

{ 
  "value": "pancakes",
  "at":"2017-04-02T14:56:30.957378Z"
}

Object with timestamp

{ 
  "value": {"latitude": 3.0735, "longitude": 51.45678},    
  "at": "2017-04-02T14:56:30.957378Z"
}

Device states
Device state assembles multiple asset states in a single message. For example, a multipurpose device can measure multiple conditions and report them to the cloud as a single message in JSON, CBOR or custom binary format.

The Cloud expects device states to arrive through HTTP or MQTT at device/<DeviceID>/state, or over LPWAN networks.

→ Read more about Device state API

JSON format
An example device state that groups temperature and daylight sensor data and their timestamps, could look like this

{
  "t": {
     "value": 33,
     "at": "2018-03-28T12:00Z+00:00"  
  },
  "dl": {
    "value": true,
    "at": "2018-03-28T12:00Z+00:00"
  }
}

To send a device state in JSON format over HTTP to the cloud, try this:

curl -X PUT \
https://api.allthingstalk.io/device/<DeviceId>/state \
-H 'authorization: Bearer <DeviceToken>' \
-H 'content-type: application/json' \
-d '{"t": {"value": 33, "at": "2018-03-28T12:00Z+00:00"}, "dl": {"value": true, "at": "2018-03-28T12:00Z+00:00"}}'

CBOR format
To construct a device state message in CBOR format, use cbor.me to encode a simplified version of device state JSON into binary.
For example, given a device state in diagnostic notation

{
  "t": 33,
  "dl": true
}

… encoded to CBOR gives this binary array: A2 61 74 18 21 62 64 6C F5

To send a device state in CBOR format over HTTP to the cloud, try this:

echo -e '\xA2\x61\x74\x18\x21\x62\x64\x6C\xF5' | \
curl -X PUT \
https://api.allthingstalk.io/device/<DeviceID>/state \
-H 'authorization: Bearer <DeviceToken>' \
-H 'content-type: application/cbor' --data-binary @-

Custom binary format
ABCL gives you freedom to specify a decoding scheme for any binary payload that you send to the cloud. Follow our ABCL tutorial to get started with encoding and decoding different types of data in as little bytes as possible. 

Receiving commands

When you want to process actuation or downlink messages on your device, you can either use asset commands over MQTT or device commands over LPWAN.

Asset commands
Asset command is an instruction or signal that causes a device to perform a single actuating function. For example, when user may send an asset command to remotely turn on the valve of a garden sprinkler.

A device can retrieve asset commands over MQTT by subscribing to these topics:

  • asset/<AssetID>/command

  • device/<DeviceID>/asset/<AssetName>/command

JSON format
Once the Cloud receives a command by a user or an application, it will forward it to a subscribed client.

{ 
  "value" : "turn on",
  "at": "2018-03-28T12:00Z+00:00",
}
  • value — a command pending for execution. Type could be any JSON value, array or object that is described in the related asset profile.

  • at — the timestamp of the command as ISO 8601 formatted string. 

To receive an actuation command your MQTT client should listen for an asset command messages in JSON.

For more info and examples go to Messaging.

Device commands
Device commands assemble multiple asset commands in a single message in order to reduce time on air and support LPWAN messages in general, that may not be designed to address individual assets.
For example, you can remotely set a sprinkler intensity and schedule when to stop.

AllThingsTalk Cloud can send device commands in CBOR and in custom binary format.

The Things Network (TTN) only for now. We will support other networks soon.

CBOR format
Once the Cloud receives a command by a user or an application, it will forward it to a subscribed client in CBOR, encoded as {"assetName": <command>} using diagnostic notation. For example:

{
  "intensity": 100,
  "stopIn": 50000
}

Custom binary format
ABCL gives you freedom to specify a decoding scheme for any binary payload that you send from the cloud to your devices. You are free to define your own ABCL scheme to encode device commands into binary and send them over LPWAN network.
Use ABCL actuate block to format custom binary commands.

→ For more info and example device commands in custom binary format go to ABCL reference.

Did this answer your question?