A short story of the migration process for The-Things-Network v2 to v3

• Preparing the applications
• Moving devices
• Moving gateways

The community driven The things network made a major overhaul of their backend services while also integrating some commercial grade paid services. The TTN is a LoRA WAN backend routing network that gathers data using many distributed community operated LoRA WAN gateways. This allows one to build a worldwide long range low bandwidth network that’s freely usable by anyone in the covered area. Since LoRA covers large areas - due to it’s nearly 110 dB link budget - this provides a network that’s reachable nearly everywhere at least with large spreading factors. In areas with better gateway coverage higher data rates and device densities can be supported - and in addition one can also use multilateration to estimate the location of a device.

If you want to know more about LoRA in general and not only about the TTN migration process take a look at my other LoRA related blog entries.

During the major overhaul the TTN has been moved to a new software stack that allows it to scale nearly infinite - most of the services are now hosted on Amazon AWS. Unfortunately this also meant that one had to migrate gateways, devices and also perform major modifications on ones applications. This has to be done until the 1st of December in 2021 - then the old v2 stack will be shut down finally. Note that the new stack supports many interesting new features that might allow one to build even more modular applications - for example one could build an on demand near infinitly scaling solution when backhauling data using webhooks into a cloud environment such as AWS and using serverless lambdas for example - this might be interesting from two points of view. On one side a hobby project might only use a few messages per time period and thus running a public machine just to subscribe to the MQTT topic (especially if you’re renting this machine in some compute center) might be economic and resource based overkill - on the other hand the MQTT subscriptions don’t scale so if you saturate on the the maximum message rate your broker can handle this approach might also be more interesting.

In my opinion one should perform the migration in the following order:

• Prepare the applications. This is usually the most work. One might subscribe to a v2 and a v3 instance in parallel using two different MQTT sessions on different topics and parse different message formats.
• Move devices from v2 to v3. This works somewhat seamless.
• Move gateways. Gateways attached to v2 forward messages to v3 but not the other way round. So one should do this only in the end.

Preparing the applications

This is the most work that one will have to do. Of course one has to register the application again in the v3 Console. One just adds the new application and assigns an application-id - it might be helpful to assign the same name as for the v2 application but this is not necessary.

Then one has to change basic connection information:

• The new MQTT server for EU region is eu1.cloud.thethings.network. The one that is used for the given application can be seen in the MQTT tab under Integrations
• As one can also see the username now includes a tenant id that’s set to @ttn for the public the things network. This of course means that one cannot encode the username inside an URL parameter any more (in case your application did that).
• The password now is a token that one can generate randomly. Remember to store that token since the console won’t show it after initial generation and the TTN won’t store the token itself.

Now the application can connect to the MQTT broker again. Then one has to account for a change in the MQTT topics. Basically they work the same as before but now they include the v3 prefix:

• V2 topics for uplink messages: <ttn app id>/devices/<ttn device id>/up
• V3 topics for uplink messages: v3/<ttn app id>@ttn/devices/<ttn device id>/up

This is usually also simple to solve. Now comes the most tedious part. The message format has radically changed. There is no raw_payload field any more - this has been renamed; Metadata is encoded entirely different as well as the device and application information inside the JSON structure. One will have a hard time just adjusting existing code instead of writing the mappings in a custom way. A typical new message looks like the following (some information has been stripped from this dump):

{
  "end_device_ids": {
    "device_id": "XXXXXXXXXXXXXXXX",
    "application_ids": {
      "application_id": "XXXXXXXXXXXXXXXX"
    },
    "dev_eui": "XXXXXXXXXXXXXXXX",
    "join_eui": "XXXXXXXXXXXXXXXX",
    "dev_addr": "XXXXXXXXXXXXXXXX"
  },
  "correlation_ids": [
    "as:up:XXXXXXXXXXXXXXXX",
    "ns:uplink:XXXXXXXXXXXXXXXX",
    "pba:conn:up:XXXXXXXXXXXXXXXX",
    "pba:uplink:XXXXXXXXXXXXXXXX",
    "rpc:/ttn.lorawan.v3.GsNs/HandleUplink:XXXXXXXXXXXXXXXX",
    "rpc:/ttn.lorawan.v3.NsAs/HandleUplink:XXXXXXXXXXXXXXXX"
  ],
  "received_at": "YYYY-MM-DDTHH:MM:SS.888102695Z",
  "uplink_message": {
    "session_key_id": "XXXXXXXXXXXXXXXX",
    "f_port": 1,
    "f_cnt": 99,
    "frm_payload": "sPWW",
    "rx_metadata": [
      {
        "gateway_ids": {
          "gateway_id": "packetbroker"
        },
        "packet_broker": {
          "message_id": "XXXXXXXXXXXXXXXX",
          "forwarder_net_id": "000013",
          "forwarder_tenant_id": "ttnv2",
          "forwarder_cluster_id": "ttn-v2-eu-4",
          "forwarder_gateway_eui": "XXXXXXXXXXXXXXXX",
          "forwarder_gateway_id": "eui-XXXXXXXXXXXXXXXX",
          "home_network_net_id": "000013",
          "home_network_tenant_id": "ttn",
          "home_network_cluster_id": "eu1.cloud.thethings.network"
        },
        "time": "YYYY-MM-DDTHH:MM:SS.888102695Z",
        "rssi": -34,
        "channel_rssi": -34,
        "snr": 8.8,
        "location": {
          "latitude": 00.00000000,
          "longitude": 00.00000000,
          "altitude": 000
        },
        "uplink_token": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      },
      {
        "gateway_ids": {
          "gateway_id": "XXXXXXXXXXXXXXXX",
          "eui": "XXXXXXXXXXXXXXXX"
        },
        "time": "YYYY-MM-DDTHH:MM:SS.888102695Z",
        "timestamp": 0000000000,
        "rssi": -34,
        "channel_rssi": -34,
        "snr": 8.8,
        "uplink_token": "XXXXXXXXXXXXXXXXXXXXXXXX",
        "channel_index": 1
      }
    ],
    "settings": {
      "data_rate": {
        "lora": {
          "bandwidth": 125000,
          "spreading_factor": 7
        }
      },
      "coding_rate": "4/5",
      "frequency": "868300000"
    },
    "received_at": "YYYY-MM-DDTHH:MM:SS.888102695Z",
    "consumed_airtime": "0.051456s",
    "network_ids": {
      "net_id": "000013",
      "tenant_id": "ttn",
      "cluster_id": "ttn-eu1"
    }
  }
}

As one can see the raw information is now found in uplink_message.frm_payload. The fields for the device and application EUI have been renamed and moved into other substructures and the metadata that had previously been embedded directly into the message format has been distributed in a variety of different nested objects.

In the best case then connect the application with a second MQTT connection to the v3 application. Be prepared to handle duplicate messages via v2 and v3 though.

Moving devices

This is pretty simple. If one does this by hand one just has to create a new entry for each device. One has to select a few properties per device when adding manually:

• Frequency plan
• Mac version 1.0.2 for v2 devices
• PHY version PHY V1.0.2 Rev B
• Copy the following values from the v2 device:
  • Device EUI
  • Application EUI
  • Application Key
  • End device ID

As soon as the device has been added to the v3 application one can simply change the application key in the v2 console and wait till the device tires the next join procedure that will then be rejected by the v2 stack and thus be forwarded to the v3 stack transparently where it succeeds. Depending on the device there are also some implementations that require one to power cycle the device to re-join.

This is everything that’s required for the devices.

Moving gateways

This is by far the most simple part but one should only move gateways after all devices have been moved to v3. Gateway that operate on the v2 endpoint forward messages also to v3 applications to reduce the migration impact on the network coverage. v3 gateways do not forward towards v2 applications.

Simply add the gateway in the v3 console. One might use the same gateway EUI in case one later on simply wants to copy the configuration. For gateways operating on the UDP packet forwarders one should also enable shedule downlink late. One should also configure the correct frequency plan and copy the gateway server address for gateway configuration.

As soon as the gateway has been entered into the console one might simply add the v3 backend to the /opt/ttn-gateway/bin/local_conf.json when using the poly packet forwarder. In this case one just has to add the new server endpoint to gateway_conf.servers (in case you’re also operating your own local network you might already deliver packets to many different endpoints anyways):

"servers": [
		{ "server_address": "router.eu.thethings.network", "serv_port_up": 1700, "serv_port_down": 1700, "serv_enabled": true },
		{ "server_address" : "eu1.cloud.thethings.network", "serv_port_up": 1700, "serv_port_down": 1700, "serv_enabled" : true }
],

As soon as migration has been fully done one might remove the old router.eu.thethings.network entry or at least disable it.

This article is tagged:

• LoRA